-- Logs begin at Wed 2022-05-25 13:54:21 UTC, end at Wed 2022-05-25 15:18:40 UTC. -- May 25 14:39:39.586093 dv-cinder-803823-9-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 25 14:39:39.595391 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 25 14:39:39.597245 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed May 25 14:39:39 2022] *** May 25 14:39:39.597245 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 25 14:39:39.597245 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 25 14:39:39.597245 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: nodename: dv-cinder-803823-9-iscsi May 25 14:39:39.597245 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: machine: x86_64 May 25 14:39:39.597245 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: clock source: unix May 25 14:39:39.597245 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: pcre jit disabled May 25 14:39:39.597245 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: detected number of CPU cores: 4 May 25 14:39:39.597245 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: current working directory: / May 25 14:39:39.597245 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: detected binary path: /usr/bin/uwsgi-core May 25 14:39:39.597245 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: your processes number limit is 47680 May 25 14:39:39.597245 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: your memory page size is 4096 bytes May 25 14:39:39.597245 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: detected max file descriptor number: 2048 May 25 14:39:39.597245 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: lock engine: pthread robust mutexes May 25 14:39:39.598635 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: thunder lock: enabled May 25 14:39:39.598635 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 25 14:39:39.598635 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 25 14:39:39.644847 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: Python main interpreter initialized at 0x559fbd6679c0 May 25 14:39:39.644847 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: python threads support enabled May 25 14:39:39.644847 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: your server socket listen backlog is limited to 100 connections May 25 14:39:39.644847 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: your mercy for graceful operations on workers is 90 seconds May 25 14:39:39.645379 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: mapped 403077 bytes (393 KB) for 2 cores May 25 14:39:39.645379 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: *** Operational MODE: preforking *** May 25 14:39:39.645752 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: *** uWSGI is running in multiple interpreter mode *** May 25 14:39:39.645752 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: spawned uWSGI master process (pid: 86677) May 25 14:39:39.646040 dv-cinder-803823-9-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 25 14:39:39.647671 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: spawned uWSGI worker 1 (pid: 86692, cores: 1) May 25 14:39:39.647671 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: spawned uWSGI worker 2 (pid: 86693, cores: 1) May 25 14:39:39.647671 dv-cinder-803823-9-iscsi devstack@keystone.service[86677]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 25 14:39:45.642507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 14:39:45.642507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: from cryptography.utils import int_from_bytes May 25 14:39:45.643763 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 14:39:45.643763 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: from cryptography.utils import int_from_bytes May 25 14:39:45.643998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 14:39:45.643998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: from cryptography.utils import int_from_bytes May 25 14:39:45.643998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 14:39:45.643998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: from cryptography.utils import int_from_bytes May 25 14:39:45.695573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 25 14:39:45.695836 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 25 14:39:45.695977 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 25 14:39:45.696148 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 25 14:39:45.696292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] command line args: [] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 25 14:39:45.696426 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] command line args: [] {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 25 14:39:45.696542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 25 14:39:45.696661 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 25 14:39:45.696778 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ================================================================================ {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 25 14:39:45.696903 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ================================================================================ {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 25 14:39:45.697001 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] admin_token = **** {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.697117 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] admin_token = **** {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.697234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] config_dir = [] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.697351 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] config_dir = [] {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.697467 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.697467 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] config_source = [] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.697678 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.697792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] debug = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.697906 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] config_source = [] {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.698050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.698639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] debug = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.698639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.699007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.699155 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.699261 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.699394 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.699522 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.699617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.699715 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.699812 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.699923 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] list_limit = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.700015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] list_limit = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.700112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] log_config_append = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.700210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] log_config_append = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.700307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.700448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.700593 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] log_dir = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.700699 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] log_dir = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.700805 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] log_file = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.700904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] log_file = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.701042 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.701166 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.701340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.701455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.701606 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.701715 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.701831 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.701954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.702138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.702262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.702381 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.702549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.702663 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.702821 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.702949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.703069 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.703184 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.703304 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.703428 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.703554 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.703692 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.703811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.703991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.704114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.704303 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.704457 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.704590 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.704709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.704709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.704920 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.705039 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.78/identity {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.705163 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.78/identity {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.705280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] publish_errors = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.705404 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] publish_errors = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.705522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.705625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.705735 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.705839 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.705962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.706110 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.706230 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.706441 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.706571 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.706696 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.706826 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] standard_threads = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.706945 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] standard_threads = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.707060 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.707179 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.707289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.707401 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.707532 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.707664 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.707782 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] use_journal = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.707898 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] use_journal = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.708051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] use_json = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.708166 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] use_json = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.708301 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] use_stderr = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.708420 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] use_stderr = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.708544 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] use_syslog = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.708665 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] use_syslog = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.708786 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.708914 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:39:45.709053 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.709176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.709291 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.709401 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.709526 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.709638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.709766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.709882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.710050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.710187 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.710311 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.710540 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.710660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.710791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.710913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.711030 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.711143 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.711273 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.711397 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.711524 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.711646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.711762 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.711886 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.712025 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.712253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.712383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.712501 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.712629 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.712744 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.712871 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.712997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.713119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.713242 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.713365 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.713477 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.713602 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.713729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.713845 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.713966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.714139 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.714271 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.714407 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.714530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.714653 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.714769 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.714892 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.715007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.715138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.715253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.715253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.715476 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.715600 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.715719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.715859 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.715974 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.716089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.716207 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.716324 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.716455 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.716685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.716810 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.716942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.717068 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.717192 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.717314 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.717449 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.717573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.717699 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.717822 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.717956 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.718116 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.718255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.718377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.718506 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.718630 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.718753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.718893 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.719020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.719142 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.719268 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.719388 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.719508 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.719649 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.719775 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.719911 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.720040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.720173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.720303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.720486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.720657 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.720825 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.720992 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.721154 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.721314 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.721486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.721644 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.721808 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.721975 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.722173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.722352 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.722553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.722681 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.722897 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.723038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.723220 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.723909 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.724111 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.724241 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.724367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.724555 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.724743 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.724925 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.725065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.725243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.725417 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.725538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.725715 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.725895 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.726080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.726213 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.726342 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.726538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.726716 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.726899 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.727092 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.727227 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.727373 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.727576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.727777 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.727906 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.728028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.728154 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.728294 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.728418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.728543 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.728669 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.728794 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.728927 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.729064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.729196 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.729383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.729514 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.729514 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.729750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.729887 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.730040 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.730181 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.730307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.730434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.730561 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.730703 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.730844 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.730973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.731100 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.731235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.731364 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.731600 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.731736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.731866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.731989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.732123 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.732255 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.732398 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.732524 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.732648 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.732773 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.732915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.733056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.733193 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.733396 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.733528 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.733660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.733841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.734163 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.734298 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.734444 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.734572 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.734700 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.734835 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.734967 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.735112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.735256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.735382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.735510 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.735637 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.735775 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.735904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.736043 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.736164 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.736299 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.736425 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.736551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.736664 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.736802 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.736926 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.737054 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.737204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.737338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.737467 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.737607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.737737 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.737867 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.737997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.738147 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.738274 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.738429 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.738561 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] auth.external = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.738561 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.738801 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.738929 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.739062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.739734 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.739867 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] auth.external = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.740038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] auth.password = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.740180 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.740306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] auth.token = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.740434 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.740568 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.740716 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.740946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.741083 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] auth.password = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.741211 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.741340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] auth.token = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.741482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.741614 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.741759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.741889 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.742067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.742200 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.742331 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.742481 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.742612 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] credential.caching = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.742742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.742867 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.742992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.743115 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.743251 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.743388 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.743514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] credential.caching = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.743637 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.743762 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.743886 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.744023 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.744151 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.744284 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.744409 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.744535 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.744667 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.744808 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.744935 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.745062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.745197 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.745311 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.745436 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.745574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.745718 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.745844 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.745980 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.746133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.746281 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.746281 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.746430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.746490 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] federation.caching = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.746548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.746612 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.746676 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.746743 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.746802 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.746860 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.746917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.746980 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.747039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] federation.caching = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.747181 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.747257 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.747320 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.747381 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.747447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.747515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.747598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.747671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.747742 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.747812 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.747882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.747947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.748013 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.748073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.748130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] identity.caching = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.748188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.748244 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.748301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.748366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.748425 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.748483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.748540 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.748597 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.748654 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.caching = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.748719 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.748778 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.748836 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.748893 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.748950 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.749007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.749072 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.749132 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.749189 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.749246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.749304 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.749360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.749425 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.749485 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.749548 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.749548 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.749652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.749717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.749782 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.749846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.749910 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.749969 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.750046 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.750046 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.750169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.750228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.750285 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.750285 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.750387 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.750453 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.750513 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.750570 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.750627 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.750684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.750741 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.750809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.750874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.750933 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.750989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.751053 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.751112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.751177 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.751236 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.751294 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.751351 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.751408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.751464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.751532 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.751592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.751649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.751713 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.751771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.751828 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.751893 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.751952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.752009 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.752071 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.752130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.752187 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.752252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.752311 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.752369 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.752426 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.752484 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.752541 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.752605 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.752665 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.752722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.752780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.752837 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.752894 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.752959 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.753018 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.753076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.753139 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.753197 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.753255 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.753255 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.753366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.753434 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.753493 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.753550 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.753607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.753672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.753731 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.753788 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.753845 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.753901 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.753958 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.754039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.754102 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.754169 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.754235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.user = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.754293 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.754351 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.754416 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.754475 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.754532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.754590 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.754647 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.754723 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.754792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.754852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.754910 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.754967 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.755030 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.755096 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.755162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.755227 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.755285 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.755348 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.755406 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.755471 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.755536 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.755594 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.755651 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.755752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.755818 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.755885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.755945 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.756003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.756060 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.756117 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.756174 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.756239 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.756456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.756639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.756639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.756866 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.756986 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.757134 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.757255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.757376 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.757496 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.757621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.757741 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.757866 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.757987 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.758179 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.758300 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.758421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.758541 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.758675 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.758797 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.758928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.759048 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.759168 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.759284 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.759415 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.759534 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.759655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.759772 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.759894 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.760015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.760151 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.760272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.760393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.760393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.760622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.760743 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.760871 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.760989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.761127 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.761207 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.761363 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.761484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.761682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] resource.caching = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.761805 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.761930 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.762078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.762214 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.762353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.762491 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.762619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.762744 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.762870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] resource.caching = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.762995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.762995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.762995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.762995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.763469 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.763657 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.763848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.764036 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.764238 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.764521 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.764651 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] role.caching = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.764773 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.764898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] role.driver = sql {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.765020 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.765152 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.765275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.765399 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.765523 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.765642 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.765764 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] role.caching = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.765896 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.766043 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] role.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.766184 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.766315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.766438 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.766559 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.766700 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.766819 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.766943 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.767063 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.767181 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.767301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.767431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.767550 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.767670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.767788 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.767919 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.768047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.768168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.768291 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.768409 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.768530 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.768530 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.768771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.768771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.768997 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.769115 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.769233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.769352 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.769488 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.769609 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.769732 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.769850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.769969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.770068 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.770251 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.770395 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.770559 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.770681 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.770805 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.770929 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.771060 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.771189 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.771317 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.771438 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.771438 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.771688 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.771817 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.771937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.772059 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.772178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.772298 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.772419 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.772548 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.772667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.772799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.772918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.773039 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.773157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.773288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] token.caching = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.773406 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.773527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.773647 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.773770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.773893 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.774131 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.774255 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.774376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.774496 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.774627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.774744 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] token.caching = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.774901 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.775019 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.775140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.775259 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.775384 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.775514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.775638 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.775755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.775874 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.775994 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.776111 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.776238 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.776365 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.776484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.776602 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.776722 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.776841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.776970 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.777087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.777205 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.777327 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.777447 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.777567 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.777695 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.777820 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] database.connection = **** {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.777955 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.778125 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.778247 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.778387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.778505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.778626 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.778756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.778874 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.778995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.779117 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.779236 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.connection = **** {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.779366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.779499 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.779589 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.779652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.779711 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.779769 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.779826 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.779884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.780086 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.780206 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.780334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.780457 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.780581 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.780704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.780834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.780957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.781041 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.781210 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.781332 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.781456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.781659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.781659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.781884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.782026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.782154 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.782230 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86692) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 25 14:39:45.782359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.782359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.782359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.782359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.782359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.782668 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:39:45.782786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 25 14:39:45.814993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:45.815201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:45.815353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:45.815439 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:45.815592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:45.815678 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:45.824900 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:45.825274 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:45.857704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:45.858054 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:45.858197 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:45.858335 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:45.919172 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:45.919679 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:45.974816 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:45.975142 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:45.976123 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:45.976357 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.046617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.047007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.047218 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.047424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.047624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.047827 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.048049 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.048255 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.048459 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.048657 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.048848 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.049069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.072826 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.073083 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.073254 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.073477 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.073696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.073909 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.074158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.074417 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.074655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.074881 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.075103 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.075318 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.075532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.075744 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.086992 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.087554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.101298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.101643 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.101856 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.102102 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.102328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.102560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.102560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.103088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.103311 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.103545 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.103761 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.103961 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.104161 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.104393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86692) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:39:46.120623 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.120954 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 14:39:46.121122 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.121122 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 14:39:46.121473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.121473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 14:39:46.121839 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.122044 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.122220 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 14:39:46.122417 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.122614 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.122785 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.122965 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.123171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.123333 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.123333 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.133710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.134066 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.134249 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.134391 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.134532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.134674 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.134812 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.134967 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.142899 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.143134 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86692) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.146866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86692) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.147038 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.147170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.147331 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.147465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.147584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.147715 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.147920 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.148072 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.148203 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.148333 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.148465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.148596 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.148742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.148873 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.149002 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.149130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.149257 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.149377 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.149515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.149644 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.149772 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.177296 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.177476 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.179026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86692) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.179194 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.179340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.179482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.184693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.185115 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.185267 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86692) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.185407 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.185626 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.185909 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.192754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.193042 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.193485 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86692) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.193776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.197829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.198215 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.198443 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.198601 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86692) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.198717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.199004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.199139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.199287 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.199420 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.199554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.199664 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.199786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.199886 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.200129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.200397 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.200668 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.219613 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.219911 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.224051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.224354 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.225143 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.228876 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.229172 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.229313 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86692) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.233019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.233319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.234397 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.238137 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.238703 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86692) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.242183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.242431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86692) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.242581 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.242770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.243066 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.243351 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.243640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.246345 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.246648 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.246934 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.247218 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.247504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.247776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.263411 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.263821 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.264096 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.264366 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.267393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86692) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.267690 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.267951 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.268218 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.273039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.274829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.275102 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.276959 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.278251 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.278543 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.278726 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.278849 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.279047 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.279158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.279382 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.282070 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86692) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.282353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.282632 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.282909 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.283181 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.293828 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.294142 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.294429 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.294724 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.295008 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.295288 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.295559 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.297301 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86692) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.297595 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.297878 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.298186 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.298464 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.298745 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.299032 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.313771 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.317635 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.318720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86692) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.321080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.322054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86692) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.322746 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.324396 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.324681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.324969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.325402 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.327056 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.328708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86692) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.328993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.329276 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.334395 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.336073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.338136 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.339394 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.339690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.339826 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86692) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.339956 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.340246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.343353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.343647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.343936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.344204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.350162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.350459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.350748 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.354874 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86692) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.355169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.355570 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.358758 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.359045 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.359334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.359615 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.359890 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.360172 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.360456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.360738 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.361019 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.361291 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.361569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.361840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.362136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.362421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.363474 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86692) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:39:46.363757 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.364101 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.364884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.365156 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.365425 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.365719 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.365984 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.366285 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.366560 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.366825 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.367082 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.367349 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.367612 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86692) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:39:46.401087 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x559fbd6679c0 pid: 86693 (default app) May 25 14:39:46.404691 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4f50b-1792-4c7c-be36-c5f663111a26 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:46.404987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4f50b-1792-4c7c-be36-c5f663111a26 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:46.405268 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4f50b-1792-4c7c-be36-c5f663111a26 None None] PATH_INFO: `/v3` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:46.407637 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x559fbd6679c0 pid: 86692 (default app) May 25 14:39:46.409614 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1/1] 10.222.0.78 () {50 vars in 791 bytes} [Wed May 25 14:39:46 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:54.951110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1415400-c729-4c1a-9c43-0c1247f5704c None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:54.951639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1415400-c729-4c1a-9c43-0c1247f5704c None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:54.951639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1415400-c729-4c1a-9c43-0c1247f5704c None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:54.956205 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1/2] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:39:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:39:54.962992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046bd68-abc1-4f25-900e-f12637fa0928 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:54.963592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046bd68-abc1-4f25-900e-f12637fa0928 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:54.963592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046bd68-abc1-4f25-900e-f12637fa0928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:55.093542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: INFO dbcounter [None req-e046bd68-abc1-4f25-900e-f12637fa0928 None None] Registered counter for database keystone May 25 14:39:55.130227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG oslo_db.sqlalchemy.engines [None req-e046bd68-abc1-4f25-900e-f12637fa0928 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86693) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 14:39:55.137422 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writer thread running {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 14:39:55.447874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG passlib.handlers.bcrypt [None req-e046bd68-abc1-4f25-900e-f12637fa0928 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86693) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 25 14:39:55.448297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG passlib.handlers.bcrypt [None req-e046bd68-abc1-4f25-900e-f12637fa0928 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86693) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 25 14:39:55.508029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-e046bd68-abc1-4f25-900e-f12637fa0928 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:39:55.509706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e046bd68-abc1-4f25-900e-f12637fa0928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:39:55.661616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 2/3] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:39:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 700 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 25 14:39:55.839795 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1415400-c729-4c1a-9c43-0c1247f5704c None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:55.915309 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: INFO dbcounter [None req-f1415400-c729-4c1a-9c43-0c1247f5704c None None] Registered counter for database keystone May 25 14:39:55.936474 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG oslo_db.sqlalchemy.engines [None req-f1415400-c729-4c1a-9c43-0c1247f5704c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86692) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 14:39:55.944370 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writer thread running {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 14:39:56.073722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1415400-c729-4c1a-9c43-0c1247f5704c None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:56.078653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9436f-09d4-400c-b47b-6e93be2f693d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:56.079354 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9436f-09d4-400c-b47b-6e93be2f693d admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:56.079642 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9436f-09d4-400c-b47b-6e93be2f693d admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:56.079922 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9436f-09d4-400c-b47b-6e93be2f693d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:56.092691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9436f-09d4-400c-b47b-6e93be2f693d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:56.166182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5fb9436f-09d4-400c-b47b-6e93be2f693d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 25 14:39:56.166562 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9436f-09d4-400c-b47b-6e93be2f693d admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:56.183251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 2/4] 10.222.0.78 () {58 vars in 1324 bytes} [Wed May 25 14:39:55 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 14:39:56.193537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046bd68-abc1-4f25-900e-f12637fa0928 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:56.221270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046bd68-abc1-4f25-900e-f12637fa0928 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:56.229857 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a6dc8-4243-4719-9207-860c015643bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:56.231985 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a6dc8-4243-4719-9207-860c015643bf admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:56.232518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a6dc8-4243-4719-9207-860c015643bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:56.233048 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a6dc8-4243-4719-9207-860c015643bf admin admin] PATH_INFO: `/v3/projects/86b029bd8d8948dfab182a990de56b86` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:56.236092 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a6dc8-4243-4719-9207-860c015643bf admin admin] RBAC: Authorizing `identity:get_project(project_id=86b029bd8d8948dfab182a990de56b86)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:56.354684 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: WARNING keystone.common.rbac_enforcer.enforcer [None req-257a6dc8-4243-4719-9207-860c015643bf admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 25 14:39:56.355233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a6dc8-4243-4719-9207-860c015643bf admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:56.365797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 3/5] 10.222.0.78 () {56 vars in 1207 bytes} [Wed May 25 14:39:56 2022] GET /identity/v3/projects/86b029bd8d8948dfab182a990de56b86 => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:39:58.674949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d4908-f5fb-425e-99c7-1f8a77bcea95 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:58.675757 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d4908-f5fb-425e-99c7-1f8a77bcea95 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:58.675903 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d4908-f5fb-425e-99c7-1f8a77bcea95 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:58.680747 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 3/6] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:39:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:39:58.687091 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5fd0d-a535-40cb-9bb9-71f39105ea18 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:58.687630 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5fd0d-a535-40cb-9bb9-71f39105ea18 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:58.687768 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5fd0d-a535-40cb-9bb9-71f39105ea18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:58.708886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-2ab5fd0d-a535-40cb-9bb9-71f39105ea18 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:39:58.710519 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2ab5fd0d-a535-40cb-9bb9-71f39105ea18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:39:58.719248 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 4/7] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:39:58 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 25 14:39:58.743165 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d4908-f5fb-425e-99c7-1f8a77bcea95 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:58.756888 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d4908-f5fb-425e-99c7-1f8a77bcea95 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:58.761446 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cd275-610b-4be2-88aa-d5f715c551bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:58.762197 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cd275-610b-4be2-88aa-d5f715c551bc admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:58.762531 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cd275-610b-4be2-88aa-d5f715c551bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:58.762854 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cd275-610b-4be2-88aa-d5f715c551bc admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:58.773058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cd275-610b-4be2-88aa-d5f715c551bc admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:58.776233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cd275-610b-4be2-88aa-d5f715c551bc admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:58.793831 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 4/8] 10.222.0.78 () {58 vars in 1324 bytes} [Wed May 25 14:39:58 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 14:39:58.803950 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab5fd0d-a535-40cb-9bb9-71f39105ea18 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:39:58.822466 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab5fd0d-a535-40cb-9bb9-71f39105ea18 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:39:58.837571 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69447d6d-534f-4829-90ce-c0f9042b984f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:39:58.839292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69447d6d-534f-4829-90ce-c0f9042b984f admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:39:58.839809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69447d6d-534f-4829-90ce-c0f9042b984f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:39:58.840314 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69447d6d-534f-4829-90ce-c0f9042b984f admin admin] PATH_INFO: `/v3/users/f27fbbbc33b24630948a570ccae5fad0` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:39:58.843086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69447d6d-534f-4829-90ce-c0f9042b984f admin admin] RBAC: Authorizing `identity:get_user(user_id=f27fbbbc33b24630948a570ccae5fad0)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:39:58.846743 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69447d6d-534f-4829-90ce-c0f9042b984f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:39:58.855773 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 5/9] 10.222.0.78 () {56 vars in 1198 bytes} [Wed May 25 14:39:58 2022] GET /identity/v3/users/f27fbbbc33b24630948a570ccae5fad0 => generated 247 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:01.278307 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58541f5c-3ed9-4d9d-afa5-689cc59ca7eb None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:01.278865 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58541f5c-3ed9-4d9d-afa5-689cc59ca7eb None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:01.278865 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58541f5c-3ed9-4d9d-afa5-689cc59ca7eb None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:01.280239 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 5/10] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:01.285931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e7410-c167-4ab5-b88b-9448fe0dd5d7 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:01.286503 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e7410-c167-4ab5-b88b-9448fe0dd5d7 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:01.286660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e7410-c167-4ab5-b88b-9448fe0dd5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:01.304790 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-671e7410-c167-4ab5-b88b-9448fe0dd5d7 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:01.306002 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-671e7410-c167-4ab5-b88b-9448fe0dd5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:01.348505 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62ec46-15ca-4ce9-867d-0be3ccdf2cbb None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:01.348956 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62ec46-15ca-4ce9-867d-0be3ccdf2cbb None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:01.349433 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62ec46-15ca-4ce9-867d-0be3ccdf2cbb None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:01.352467 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 6/11] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:40:01.358891 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8734a55-a318-48be-8655-be66f04d60ec None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:01.359304 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8734a55-a318-48be-8655-be66f04d60ec None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:01.359715 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8734a55-a318-48be-8655-be66f04d60ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:01.422992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 6/12] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:01.423412 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG passlib.handlers.bcrypt [None req-c8734a55-a318-48be-8655-be66f04d60ec None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86692) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 25 14:40:01.424051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG passlib.handlers.bcrypt [None req-c8734a55-a318-48be-8655-be66f04d60ec None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86692) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 25 14:40:01.447533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e7410-c167-4ab5-b88b-9448fe0dd5d7 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:01.475558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-c8734a55-a318-48be-8655-be66f04d60ec None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:01.478032 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-c8734a55-a318-48be-8655-be66f04d60ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:01.486937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:01.606646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e7410-c167-4ab5-b88b-9448fe0dd5d7 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:01.611779 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 7/13] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:40:01.652643 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8734a55-a318-48be-8655-be66f04d60ec None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:01.666647 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4bce3c-0116-49ae-933b-24491ba8000d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:01.667443 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4bce3c-0116-49ae-933b-24491ba8000d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:01.667790 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4bce3c-0116-49ae-933b-24491ba8000d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:01.668135 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4bce3c-0116-49ae-933b-24491ba8000d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:01.763463 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4bce3c-0116-49ae-933b-24491ba8000d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:01.765088 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4bce3c-0116-49ae-933b-24491ba8000d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:01.795314 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:01.894315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8734a55-a318-48be-8655-be66f04d60ec None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:01.898757 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:01.908824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 7/14] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:40:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 14:40:01.915724 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4bce3c-0116-49ae-933b-24491ba8000d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:01.940128 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116fa39-b923-4d11-af96-fe2b9cded947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:01.941203 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116fa39-b923-4d11-af96-fe2b9cded947 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:01.941521 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116fa39-b923-4d11-af96-fe2b9cded947 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:01.941798 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116fa39-b923-4d11-af96-fe2b9cded947 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:01.955473 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116fa39-b923-4d11-af96-fe2b9cded947 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:01.968250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-5116fa39-b923-4d11-af96-fe2b9cded947 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:40:01.968250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:01.968250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:01.968250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:01.968250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:01.968250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:01.968250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:01.968250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:01.968250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:01.968250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:01.968250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:01.968250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:01.968250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:01.968250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:01.968250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:01.968250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:01.969251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:01.969251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:01.969251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:01.969251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:01.969251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:01.969251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:01.969251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:01.969251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:01.969251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:01.969251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:01.969251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:01.969251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:40:01.969251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:01.969251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:01.969251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:01.969251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:01.970272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:01.970272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:01.970272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:01.970272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:01.970272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:01.970272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:01.970272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:01.970272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:01.970272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:01.970272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:01.970272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:40:01.970272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:40:01.970272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 8/15] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:40:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:40:01.976177 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116fa39-b923-4d11-af96-fe2b9cded947 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.035917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4bce3c-0116-49ae-933b-24491ba8000d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.067965 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:02.081975 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46739b-6c6b-4948-8025-b62409877eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.082798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46739b-6c6b-4948-8025-b62409877eb0 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.083130 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46739b-6c6b-4948-8025-b62409877eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.083385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46739b-6c6b-4948-8025-b62409877eb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.093258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116fa39-b923-4d11-af96-fe2b9cded947 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.094672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46739b-6c6b-4948-8025-b62409877eb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.096199 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46739b-6c6b-4948-8025-b62409877eb0 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.121150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-9b46739b-6c6b-4948-8025-b62409877eb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:40:02.121150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:02.121150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:02.121150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:02.121150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:02.121150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:02.121150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:02.121150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:02.121150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:02.121150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:02.121150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:02.121150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:02.121150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:40:02.121150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:40:02.121150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:40:02.122987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:40:02.122987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:02.122987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:02.122987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:02.122987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:02.122987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:02.122987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:02.122987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:02.122987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:40:02.122987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:02.122987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:02.122987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:02.122987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:02.122987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:02.122987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:02.122987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:02.122987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:02.124312 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:40:02.124312 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:40:02.124312 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:40:02.124312 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:40:02.124312 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 8/16] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:40:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:40:02.129353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46739b-6c6b-4948-8025-b62409877eb0 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.132347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a712b4-a58b-4650-9224-d40c7332731d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.133095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a712b4-a58b-4650-9224-d40c7332731d None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.133389 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a712b4-a58b-4650-9224-d40c7332731d None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.133668 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a712b4-a58b-4650-9224-d40c7332731d None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.134564 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a712b4-a58b-4650-9224-d40c7332731d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:02.134871 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a712b4-a58b-4650-9224-d40c7332731d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.136353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a712b4-a58b-4650-9224-d40c7332731d None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.149440 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 9/17] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:02.154864 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a712b4-a58b-4650-9224-d40c7332731d None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.186926 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:02.241487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46739b-6c6b-4948-8025-b62409877eb0 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.276947 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a712b4-a58b-4650-9224-d40c7332731d None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.285961 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe9e03-4f69-4b74-b6f3-cb73901eb888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.287033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe9e03-4f69-4b74-b6f3-cb73901eb888 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.287462 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe9e03-4f69-4b74-b6f3-cb73901eb888 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.287859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe9e03-4f69-4b74-b6f3-cb73901eb888 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.288784 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe9e03-4f69-4b74-b6f3-cb73901eb888 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:02.289219 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe9e03-4f69-4b74-b6f3-cb73901eb888 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.291121 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe9e03-4f69-4b74-b6f3-cb73901eb888 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.310996 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 9/18] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:02.314365 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9660f-4741-4b66-b8ba-731847bf4807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.315273 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9660f-4741-4b66-b8ba-731847bf4807 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.315555 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9660f-4741-4b66-b8ba-731847bf4807 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.315827 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9660f-4741-4b66-b8ba-731847bf4807 None admin] PATH_INFO: `/v3/users/f27fbbbc33b24630948a570ccae5fad0` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.317633 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9660f-4741-4b66-b8ba-731847bf4807 None admin] RBAC: Authorizing `identity:get_user(user_id=f27fbbbc33b24630948a570ccae5fad0)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.319205 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9660f-4741-4b66-b8ba-731847bf4807 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.321277 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 10/19] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:02 2022] GET /identity/v3/users/f27fbbbc33b24630948a570ccae5fad0 => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:02.327218 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe9e03-4f69-4b74-b6f3-cb73901eb888 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.371091 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:02.455588 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe9e03-4f69-4b74-b6f3-cb73901eb888 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.512593 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b13221-7408-4ba8-a3ce-6713a28e20c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.513472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b13221-7408-4ba8-a3ce-6713a28e20c7 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.513761 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b13221-7408-4ba8-a3ce-6713a28e20c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.514069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b13221-7408-4ba8-a3ce-6713a28e20c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.515333 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b13221-7408-4ba8-a3ce-6713a28e20c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.516866 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b13221-7408-4ba8-a3ce-6713a28e20c7 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.518888 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 10/20] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:40:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:02.528356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9660f-4741-4b66-b8ba-731847bf4807 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.665399 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:02.673180 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9660f-4741-4b66-b8ba-731847bf4807 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.720629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bcd172-1c36-49ab-b347-e0368e1601eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.721238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcd172-1c36-49ab-b347-e0368e1601eb None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.721524 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcd172-1c36-49ab-b347-e0368e1601eb None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.721799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcd172-1c36-49ab-b347-e0368e1601eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.723237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bcd172-1c36-49ab-b347-e0368e1601eb None admin] RBAC: Adding query filter params (role.id=e566b37cd92f467198b5dc40f183effc, scope.domain.id=default, user.id=f27fbbbc33b24630948a570ccae5fad0) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:02.723563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bcd172-1c36-49ab-b347-e0368e1601eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.725077 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bcd172-1c36-49ab-b347-e0368e1601eb None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.735304 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 11/21] 10.222.0.78 () {56 vars in 1322 bytes} [Wed May 25 14:40:02 2022] GET /identity/v3/role_assignments?user.id=f27fbbbc33b24630948a570ccae5fad0&scope.domain.id=default&role.id=e566b37cd92f467198b5dc40f183effc => generated 233 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:05.376107 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbeda8d-e9e3-4c6d-9e74-5c96a6d49982 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:05.377130 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbeda8d-e9e3-4c6d-9e74-5c96a6d49982 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:05.377622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbeda8d-e9e3-4c6d-9e74-5c96a6d49982 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:05.380414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 11/22] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:05.388832 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e3ce4-d126-42a4-8a9b-5298425a3317 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:05.389799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e3ce4-d126-42a4-8a9b-5298425a3317 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:05.390316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e3ce4-d126-42a4-8a9b-5298425a3317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:05.413417 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-7f1e3ce4-d126-42a4-8a9b-5298425a3317 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:05.415333 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-7f1e3ce4-d126-42a4-8a9b-5298425a3317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:05.507481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 12/23] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:05.546547 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbeda8d-e9e3-4c6d-9e74-5c96a6d49982 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:05.688084 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbeda8d-e9e3-4c6d-9e74-5c96a6d49982 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:05.690169 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:05.736669 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb662d7-7e38-4d6f-8169-c33f26a732d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:05.737870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb662d7-7e38-4d6f-8169-c33f26a732d4 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:05.738942 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb662d7-7e38-4d6f-8169-c33f26a732d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:05.739416 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb662d7-7e38-4d6f-8169-c33f26a732d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:05.756591 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb662d7-7e38-4d6f-8169-c33f26a732d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:05.758538 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb662d7-7e38-4d6f-8169-c33f26a732d4 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:05.772633 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db3a11-6229-4042-a090-b8744fe51811 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:05.773248 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db3a11-6229-4042-a090-b8744fe51811 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:05.773728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db3a11-6229-4042-a090-b8744fe51811 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:05.775711 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 13/24] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:05.783819 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb0cfd-1f0e-4f36-8bfa-afaff7e19d4e None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:05.784490 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb0cfd-1f0e-4f36-8bfa-afaff7e19d4e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:05.784970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb0cfd-1f0e-4f36-8bfa-afaff7e19d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:05.792505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-0eb662d7-7e38-4d6f-8169-c33f26a732d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:40:05.792505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:05.792505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:05.792505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:05.792505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:05.792505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:05.792505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:05.792505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:05.792505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:05.792505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:05.792505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:05.792505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:05.792505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:40:05.792505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:40:05.792505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:40:05.794383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:40:05.794383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:05.794383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:05.794383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:05.794383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:05.794383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:05.794383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:05.794383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:05.794383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:40:05.794383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:05.794383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:05.794383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:05.794383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:05.794383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:05.794383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:05.794383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:05.794383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:05.795342 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:40:05.795342 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:40:05.795342 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:40:05.795342 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:40:05.795342 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 12/25] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:40:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:05.805507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb662d7-7e38-4d6f-8169-c33f26a732d4 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:05.828102 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-94bb0cfd-1f0e-4f36-8bfa-afaff7e19d4e None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:05.830247 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-94bb0cfd-1f0e-4f36-8bfa-afaff7e19d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:06.059071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 14/26] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:06.061676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d2309-60f4-479b-adcd-2a9f5e1c80b9 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:06.062224 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d2309-60f4-479b-adcd-2a9f5e1c80b9 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:06.062725 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d2309-60f4-479b-adcd-2a9f5e1c80b9 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:06.072451 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 15/27] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:06 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:06.083118 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d354a-6751-4c03-bc89-32b21f3175ba None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:06.083626 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d354a-6751-4c03-bc89-32b21f3175ba None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:06.084131 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d354a-6751-4c03-bc89-32b21f3175ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:06.143209 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-221d354a-6751-4c03-bc89-32b21f3175ba None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:06.145813 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-221d354a-6751-4c03-bc89-32b21f3175ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:06.184974 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:06.202656 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb662d7-7e38-4d6f-8169-c33f26a732d4 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:06.292567 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6f9d4-9079-4fcd-8bb1-3b68cb8462fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:06.293661 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6f9d4-9079-4fcd-8bb1-3b68cb8462fd None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:06.294186 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6f9d4-9079-4fcd-8bb1-3b68cb8462fd None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:06.294714 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6f9d4-9079-4fcd-8bb1-3b68cb8462fd None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:06.295660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6f9d4-9079-4fcd-8bb1-3b68cb8462fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:06.296175 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6f9d4-9079-4fcd-8bb1-3b68cb8462fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:06.298071 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6f9d4-9079-4fcd-8bb1-3b68cb8462fd None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:06.322282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 16/28] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:40:06.323267 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d354a-6751-4c03-bc89-32b21f3175ba None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:06.331003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 13/29] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 527 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:06.343775 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6f9d4-9079-4fcd-8bb1-3b68cb8462fd None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:06.389201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:06.531504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d354a-6751-4c03-bc89-32b21f3175ba None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:06.560006 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6f9d4-9079-4fcd-8bb1-3b68cb8462fd None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:06.586920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51193231-c93d-4d31-beef-4c1bf5c03bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:06.588254 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51193231-c93d-4d31-beef-4c1bf5c03bf9 None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:06.589293 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51193231-c93d-4d31-beef-4c1bf5c03bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:06.590090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51193231-c93d-4d31-beef-4c1bf5c03bf9 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:06.591457 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51193231-c93d-4d31-beef-4c1bf5c03bf9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:06.594028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51193231-c93d-4d31-beef-4c1bf5c03bf9 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:06.601681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1d83c-42db-42c7-92c2-17d0d302ff28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:06.603100 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1d83c-42db-42c7-92c2-17d0d302ff28 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:06.603636 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1d83c-42db-42c7-92c2-17d0d302ff28 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:06.603867 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1d83c-42db-42c7-92c2-17d0d302ff28 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:06.605762 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1d83c-42db-42c7-92c2-17d0d302ff28 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:06.608160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1d83c-42db-42c7-92c2-17d0d302ff28 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:06.742984 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 17/30] 10.222.0.78 () {60 vars in 1131 bytes} [Wed May 25 14:40:06 2022] POST /identity/v3/roles => generated 225 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:06.744386 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51193231-c93d-4d31-beef-4c1bf5c03bf9 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:06.759344 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:06.775641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 14/31] 10.222.0.78 () {60 vars in 1141 bytes} [Wed May 25 14:40:06 2022] POST /identity/v3/projects => generated 301 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:06.776876 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae0618-0b34-473b-bb54-0fc7fef415a6 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:06.777172 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae0618-0b34-473b-bb54-0fc7fef415a6 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:06.777455 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae0618-0b34-473b-bb54-0fc7fef415a6 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:06.779567 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 15/32] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:06.797339 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371c5a0-efeb-4d62-bcf3-4a701111c22e None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:06.797791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371c5a0-efeb-4d62-bcf3-4a701111c22e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:06.798210 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371c5a0-efeb-4d62-bcf3-4a701111c22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:06.835546 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-9371c5a0-efeb-4d62-bcf3-4a701111c22e None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:06.841688 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9371c5a0-efeb-4d62-bcf3-4a701111c22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:06.885596 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51193231-c93d-4d31-beef-4c1bf5c03bf9 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:06.930370 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7853a0-062b-42e5-8c32-8dab46f7a817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:06.931486 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7853a0-062b-42e5-8c32-8dab46f7a817 None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:06.931987 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7853a0-062b-42e5-8c32-8dab46f7a817 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:06.932498 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7853a0-062b-42e5-8c32-8dab46f7a817 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:06.933518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7853a0-062b-42e5-8c32-8dab46f7a817 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:06.935474 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7853a0-062b-42e5-8c32-8dab46f7a817 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:06.954850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 18/33] 10.222.0.78 () {60 vars in 1131 bytes} [Wed May 25 14:40:06 2022] POST /identity/v3/roles => generated 219 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:06.976812 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:06.993144 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 16/34] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:40:07.037659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7853a0-062b-42e5-8c32-8dab46f7a817 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:07.128877 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:07.185315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7853a0-062b-42e5-8c32-8dab46f7a817 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:07.239067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca052ccb-cc90-466f-a44d-f47ecca756f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:07.239957 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca052ccb-cc90-466f-a44d-f47ecca756f0 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:07.240249 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca052ccb-cc90-466f-a44d-f47ecca756f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:07.240751 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca052ccb-cc90-466f-a44d-f47ecca756f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:07.260922 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca052ccb-cc90-466f-a44d-f47ecca756f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:07.268898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-ca052ccb-cc90-466f-a44d-f47ecca756f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:40:07.268898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:07.268898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:07.268898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:07.268898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:07.268898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:07.268898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:07.268898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:07.268898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:07.268898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:07.268898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:07.268898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:07.268898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:07.268898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:07.268898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:07.268898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:07.270429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:07.270429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:07.270429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:07.270429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:07.270429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:07.270429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:07.270429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:07.270429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:07.270429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:07.270429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:07.270429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:07.270429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:40:07.270429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:07.270429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:07.270429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:07.270429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:07.271212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:07.271212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:07.271212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:07.271212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:07.271212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:07.271212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:07.271212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:07.271212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:07.271212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:07.271212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:07.271212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:40:07.271212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:40:07.271212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 19/35] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:40:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:07.277726 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371c5a0-efeb-4d62-bcf3-4a701111c22e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:07.367852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476edc8-ffd6-430d-a624-ff7dda792a9a None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:07.368737 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476edc8-ffd6-430d-a624-ff7dda792a9a None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:07.369109 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476edc8-ffd6-430d-a624-ff7dda792a9a None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:07.371084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 20/36] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:07.378707 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7c091-dbe5-4d2a-a7b8-3c0237929c40 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:07.379072 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7c091-dbe5-4d2a-a7b8-3c0237929c40 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:07.379317 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7c091-dbe5-4d2a-a7b8-3c0237929c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:07.405191 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-19e7c091-dbe5-4d2a-a7b8-3c0237929c40 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:07.408414 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-19e7c091-dbe5-4d2a-a7b8-3c0237929c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:07.492178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371c5a0-efeb-4d62-bcf3-4a701111c22e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:07.536293 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 21/37] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:40:07.562787 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e7c091-dbe5-4d2a-a7b8-3c0237929c40 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:07.566772 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b3e0cc-e531-4bb9-837f-8f1e48508c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:07.567999 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3e0cc-e531-4bb9-837f-8f1e48508c3f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:07.568537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3e0cc-e531-4bb9-837f-8f1e48508c3f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:07.569044 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3e0cc-e531-4bb9-837f-8f1e48508c3f None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:07.570639 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b3e0cc-e531-4bb9-837f-8f1e48508c3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:07.571213 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b3e0cc-e531-4bb9-837f-8f1e48508c3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:07.574242 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b3e0cc-e531-4bb9-837f-8f1e48508c3f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:07.599574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 17/38] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:07.609266 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b3e0cc-e531-4bb9-837f-8f1e48508c3f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:07.671992 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:07.711256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:07.728731 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e7c091-dbe5-4d2a-a7b8-3c0237929c40 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:07.771221 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b3e0cc-e531-4bb9-837f-8f1e48508c3f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:07.775054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a0f75e-7a29-4aa4-95c0-60e48b883342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:07.775745 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0f75e-7a29-4aa4-95c0-60e48b883342 None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:07.776026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0f75e-7a29-4aa4-95c0-60e48b883342 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:07.776302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0f75e-7a29-4aa4-95c0-60e48b883342 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:07.777085 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a0f75e-7a29-4aa4-95c0-60e48b883342 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:07.778563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a0f75e-7a29-4aa4-95c0-60e48b883342 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:07.796447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 22/39] 10.222.0.78 () {60 vars in 1131 bytes} [Wed May 25 14:40:07 2022] POST /identity/v3/roles => generated 223 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:07.797836 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ea6ea-6830-47bb-b69f-da45720a7413 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:07.798162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ea6ea-6830-47bb-b69f-da45720a7413 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:07.798436 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ea6ea-6830-47bb-b69f-da45720a7413 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:07.800108 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 23/40] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:07.807238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95daa4bc-4880-4df5-8e5e-03e0ea8ede0b None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:07.807518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95daa4bc-4880-4df5-8e5e-03e0ea8ede0b None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:07.807793 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95daa4bc-4880-4df5-8e5e-03e0ea8ede0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:07.819638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d88d6-823f-4fe7-bd6d-040fcd30e726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:07.820801 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d88d6-823f-4fe7-bd6d-040fcd30e726 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:07.821239 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d88d6-823f-4fe7-bd6d-040fcd30e726 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:07.821674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d88d6-823f-4fe7-bd6d-040fcd30e726 None admin] PATH_INFO: `/v3/users/f27fbbbc33b24630948a570ccae5fad0` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:07.823433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d88d6-823f-4fe7-bd6d-040fcd30e726 None admin] RBAC: Authorizing `identity:get_user(user_id=f27fbbbc33b24630948a570ccae5fad0)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:07.825018 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d88d6-823f-4fe7-bd6d-040fcd30e726 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:07.827380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 18/41] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:07 2022] GET /identity/v3/users/f27fbbbc33b24630948a570ccae5fad0 => generated 247 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:07.829876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-95daa4bc-4880-4df5-8e5e-03e0ea8ede0b None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:07.831647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-95daa4bc-4880-4df5-8e5e-03e0ea8ede0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:07.836515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d88d6-823f-4fe7-bd6d-040fcd30e726 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:07.955316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 24/42] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:40:07.999854 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95daa4bc-4880-4df5-8e5e-03e0ea8ede0b None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:08.029519 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d88d6-823f-4fe7-bd6d-040fcd30e726 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:08.064092 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:08.073445 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:08.075695 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ea8c4-c95f-4c8c-82e6-48fc525d870d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:08.075695 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ea8c4-c95f-4c8c-82e6-48fc525d870d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:08.075695 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ea8c4-c95f-4c8c-82e6-48fc525d870d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:08.077010 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ea8c4-c95f-4c8c-82e6-48fc525d870d None admin] PATH_INFO: `/v3/domains/default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:08.078050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46ea8c4-c95f-4c8c-82e6-48fc525d870d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:08.079329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46ea8c4-c95f-4c8c-82e6-48fc525d870d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:08.081118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 19/43] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:40:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:08.087254 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ea8c4-c95f-4c8c-82e6-48fc525d870d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:08.141034 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95daa4bc-4880-4df5-8e5e-03e0ea8ede0b None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:08.196973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e0d14-c28e-4fed-b998-0e804971b333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:08.197787 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e0d14-c28e-4fed-b998-0e804971b333 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:08.198100 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e0d14-c28e-4fed-b998-0e804971b333 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:08.198377 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e0d14-c28e-4fed-b998-0e804971b333 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:08.199725 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e0d14-c28e-4fed-b998-0e804971b333 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:08.201248 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e0d14-c28e-4fed-b998-0e804971b333 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:08.203001 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 25/44] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:40:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:08.209424 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e0d14-c28e-4fed-b998-0e804971b333 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:08.209587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ea8c4-c95f-4c8c-82e6-48fc525d870d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:08.251459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d64671-3565-4a9f-8a52-900e41614716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:08.252682 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d64671-3565-4a9f-8a52-900e41614716 None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:08.252923 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d64671-3565-4a9f-8a52-900e41614716 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:08.253293 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d64671-3565-4a9f-8a52-900e41614716 None admin] PATH_INFO: `/v3/domains/default/users/f27fbbbc33b24630948a570ccae5fad0/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:08.256024 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d64671-3565-4a9f-8a52-900e41614716 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f27fbbbc33b24630948a570ccae5fad0, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:08.257549 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d64671-3565-4a9f-8a52-900e41614716 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:08.273184 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 20/45] 10.222.0.78 () {58 vars in 1361 bytes} [Wed May 25 14:40:08 2022] PUT /identity/v3/domains/default/users/f27fbbbc33b24630948a570ccae5fad0/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:40:08.354483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e0d14-c28e-4fed-b998-0e804971b333 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:08.374038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:08.394934 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bcb584-16e5-4f7f-8d1d-7bb5fc5b2b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:08.395685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bcb584-16e5-4f7f-8d1d-7bb5fc5b2b86 None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:08.395978 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bcb584-16e5-4f7f-8d1d-7bb5fc5b2b86 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:08.396404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bcb584-16e5-4f7f-8d1d-7bb5fc5b2b86 None admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:08.396967 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bcb584-16e5-4f7f-8d1d-7bb5fc5b2b86 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:08.398496 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bcb584-16e5-4f7f-8d1d-7bb5fc5b2b86 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:08.438219 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 26/46] 10.222.0.78 () {60 vars in 1141 bytes} [Wed May 25 14:40:08 2022] POST /identity/v3/projects => generated 312 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:40:10.602352 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1174e78-bc35-4d6a-b2b9-b7963a3de0e7 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:10.602944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1174e78-bc35-4d6a-b2b9-b7963a3de0e7 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:10.602944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1174e78-bc35-4d6a-b2b9-b7963a3de0e7 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:10.604512 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 21/47] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:10.611779 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d24b0-3d59-45e0-94c2-809e7bc81d51 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:10.612338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d24b0-3d59-45e0-94c2-809e7bc81d51 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:10.612863 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d24b0-3d59-45e0-94c2-809e7bc81d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:10.634833 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-0b6d24b0-3d59-45e0-94c2-809e7bc81d51 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:10.636167 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-0b6d24b0-3d59-45e0-94c2-809e7bc81d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:10.746053 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 27/48] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:10.767850 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491e9f7-4505-4cbc-9d28-ea6868e8f838 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:10.768145 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491e9f7-4505-4cbc-9d28-ea6868e8f838 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:10.768436 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491e9f7-4505-4cbc-9d28-ea6868e8f838 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:10.769931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 22/49] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:10.773193 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d24b0-3d59-45e0-94c2-809e7bc81d51 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:10.776735 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b1554-927e-4a75-812b-8aee3874da6a None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:10.777186 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b1554-927e-4a75-812b-8aee3874da6a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:10.777700 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b1554-927e-4a75-812b-8aee3874da6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:10.804306 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-898b1554-927e-4a75-812b-8aee3874da6a None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:10.806153 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-898b1554-927e-4a75-812b-8aee3874da6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:10.876875 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:10.884484 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:10.889129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d24b0-3d59-45e0-94c2-809e7bc81d51 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:10.898062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 23/50] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:10.926644 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b1554-927e-4a75-812b-8aee3874da6a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:10.931819 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35922b-0fa1-40d9-816b-5744e11a7db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:10.932627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35922b-0fa1-40d9-816b-5744e11a7db5 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:10.932911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35922b-0fa1-40d9-816b-5744e11a7db5 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:10.933186 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35922b-0fa1-40d9-816b-5744e11a7db5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:10.944315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a35922b-0fa1-40d9-816b-5744e11a7db5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:10.952168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-8a35922b-0fa1-40d9-816b-5744e11a7db5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:40:10.952168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:10.952168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:10.952168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:10.952168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:10.952168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:10.952168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:10.952168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:10.952168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:10.952168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:10.952168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:10.952168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:10.952168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:10.952168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:10.952168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:10.952168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:10.953040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:10.953040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:10.953040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:10.953040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:10.953040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:10.953040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:10.953040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:10.953040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:10.953040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:10.953040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:10.953040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:10.953040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:40:10.953040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:10.953040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:10.953040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:10.953040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:10.954033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:10.954033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:10.954033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:10.954033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:10.954033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:10.954033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:10.954033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:10.954033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:10.954033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:10.954033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:10.954033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:40:10.954033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:40:10.954033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 28/51] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:40:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:10.959299 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35922b-0fa1-40d9-816b-5744e11a7db5 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:11.040363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b1554-927e-4a75-812b-8aee3874da6a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:11.067280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35922b-0fa1-40d9-816b-5744e11a7db5 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:11.083290 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814fe12-42b7-4283-ac16-8cf2300bc04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:11.084308 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814fe12-42b7-4283-ac16-8cf2300bc04b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:11.084721 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814fe12-42b7-4283-ac16-8cf2300bc04b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:11.085074 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814fe12-42b7-4283-ac16-8cf2300bc04b None admin] PATH_INFO: `/v3/domains/default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:11.086651 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814fe12-42b7-4283-ac16-8cf2300bc04b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:11.088261 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814fe12-42b7-4283-ac16-8cf2300bc04b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:11.089923 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 24/52] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:40:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:11.095453 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814fe12-42b7-4283-ac16-8cf2300bc04b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:11.104137 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc33eb0-96e5-4020-abb1-3dcf943948ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:11.104819 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc33eb0-96e5-4020-abb1-3dcf943948ce None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:11.105112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc33eb0-96e5-4020-abb1-3dcf943948ce None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:11.105398 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc33eb0-96e5-4020-abb1-3dcf943948ce None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:11.106188 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc33eb0-96e5-4020-abb1-3dcf943948ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:11.106502 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc33eb0-96e5-4020-abb1-3dcf943948ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:11.107921 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc33eb0-96e5-4020-abb1-3dcf943948ce None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:11.116945 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 29/53] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:11.122512 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc33eb0-96e5-4020-abb1-3dcf943948ce None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:11.147463 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:11.155977 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:11.232644 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814fe12-42b7-4283-ac16-8cf2300bc04b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:11.237041 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc33eb0-96e5-4020-abb1-3dcf943948ce None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:11.287906 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf37edd-c564-4f80-8371-dc590b6e69ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:11.288699 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf37edd-c564-4f80-8371-dc590b6e69ea None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:11.289074 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf37edd-c564-4f80-8371-dc590b6e69ea None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:11.289291 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf37edd-c564-4f80-8371-dc590b6e69ea None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:11.294136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf37edd-c564-4f80-8371-dc590b6e69ea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:11.294905 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ecc7a9-2375-4a29-abdf-adedf14e0753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:11.295798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf37edd-c564-4f80-8371-dc590b6e69ea None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:11.295889 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ecc7a9-2375-4a29-abdf-adedf14e0753 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:11.295889 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ecc7a9-2375-4a29-abdf-adedf14e0753 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:11.295889 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ecc7a9-2375-4a29-abdf-adedf14e0753 None admin] PATH_INFO: `/v3/users/f27fbbbc33b24630948a570ccae5fad0` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:11.295889 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ecc7a9-2375-4a29-abdf-adedf14e0753 None admin] RBAC: Authorizing `identity:get_user(user_id=f27fbbbc33b24630948a570ccae5fad0)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:11.296551 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ecc7a9-2375-4a29-abdf-adedf14e0753 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:11.298427 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 30/54] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:11 2022] GET /identity/v3/users/f27fbbbc33b24630948a570ccae5fad0 => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:11.304259 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ecc7a9-2375-4a29-abdf-adedf14e0753 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:11.322678 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 25/55] 10.222.0.78 () {60 vars in 1140 bytes} [Wed May 25 14:40:11 2022] POST /identity/v3/projects => generated 298 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:40:11.434897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:11.455172 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ecc7a9-2375-4a29-abdf-adedf14e0753 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:11.495283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d2f93-ca04-4f7f-8c0c-ef3701662189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:11.496089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d2f93-ca04-4f7f-8c0c-ef3701662189 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:11.496372 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d2f93-ca04-4f7f-8c0c-ef3701662189 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:11.496644 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d2f93-ca04-4f7f-8c0c-ef3701662189 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:11.497912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d2f93-ca04-4f7f-8c0c-ef3701662189 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:11.499428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d2f93-ca04-4f7f-8c0c-ef3701662189 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:11.501157 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 31/56] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:40:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:11.507754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf37edd-c564-4f80-8371-dc590b6e69ea None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:11.632934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf37edd-c564-4f80-8371-dc590b6e69ea None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:11.635390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:11.671844 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90999421-9cac-4168-af05-4ccb53a5ec0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:11.672513 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90999421-9cac-4168-af05-4ccb53a5ec0b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:11.672802 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90999421-9cac-4168-af05-4ccb53a5ec0b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:11.673151 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90999421-9cac-4168-af05-4ccb53a5ec0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:11.674035 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90999421-9cac-4168-af05-4ccb53a5ec0b None admin] RBAC: Adding query filter params (role.id=e566b37cd92f467198b5dc40f183effc, scope.domain.id=default, user.id=f27fbbbc33b24630948a570ccae5fad0) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:11.674350 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90999421-9cac-4168-af05-4ccb53a5ec0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:11.676004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90999421-9cac-4168-af05-4ccb53a5ec0b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:11.684338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 26/57] 10.222.0.78 () {56 vars in 1322 bytes} [Wed May 25 14:40:11 2022] GET /identity/v3/role_assignments?user.id=f27fbbbc33b24630948a570ccae5fad0&scope.domain.id=default&role.id=e566b37cd92f467198b5dc40f183effc => generated 530 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:13.665067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c73523-414b-43b5-8130-c48c2bd8c95d None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:13.665595 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c73523-414b-43b5-8130-c48c2bd8c95d None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:13.665595 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c73523-414b-43b5-8130-c48c2bd8c95d None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:13.667037 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 32/58] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:13.672519 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112e977-7f73-45c6-9207-506b82b35aad None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:13.672962 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112e977-7f73-45c6-9207-506b82b35aad None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:13.673088 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112e977-7f73-45c6-9207-506b82b35aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:13.690462 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-3112e977-7f73-45c6-9207-506b82b35aad None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:13.691725 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3112e977-7f73-45c6-9207-506b82b35aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:13.763129 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 27/59] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:13.788630 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c73523-414b-43b5-8130-c48c2bd8c95d None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:13.910961 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c73523-414b-43b5-8130-c48c2bd8c95d None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:13.948235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b44d88-8572-4e38-8f92-544c1b9383e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:13.948958 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b44d88-8572-4e38-8f92-544c1b9383e9 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:13.949236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b44d88-8572-4e38-8f92-544c1b9383e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:13.949520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b44d88-8572-4e38-8f92-544c1b9383e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:13.950766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b44d88-8572-4e38-8f92-544c1b9383e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:13.952242 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b44d88-8572-4e38-8f92-544c1b9383e9 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:13.953903 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 33/60] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:40:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:13.959655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112e977-7f73-45c6-9207-506b82b35aad None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:14.066185 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:14.070489 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112e977-7f73-45c6-9207-506b82b35aad None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:14.108407 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2626e52-856f-4c5e-8ef0-500dc2396b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:14.109097 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2626e52-856f-4c5e-8ef0-500dc2396b1e None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:14.109384 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2626e52-856f-4c5e-8ef0-500dc2396b1e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:14.109661 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2626e52-856f-4c5e-8ef0-500dc2396b1e None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:14.110434 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2626e52-856f-4c5e-8ef0-500dc2396b1e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:14.111900 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2626e52-856f-4c5e-8ef0-500dc2396b1e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:14.139807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 28/61] 10.222.0.78 () {60 vars in 1132 bytes} [Wed May 25 14:40:13 2022] POST /identity/v3/users => generated 275 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:17.731467 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1580a-7871-4cbd-8e96-a6a27976078c None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:17.732558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1580a-7871-4cbd-8e96-a6a27976078c None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:17.733049 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1580a-7871-4cbd-8e96-a6a27976078c None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:17.735827 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 34/62] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:17.742080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767ede8-34b1-47b5-9313-1cee30e02a62 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:17.743300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767ede8-34b1-47b5-9313-1cee30e02a62 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:17.744031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767ede8-34b1-47b5-9313-1cee30e02a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:17.772013 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-e767ede8-34b1-47b5-9313-1cee30e02a62 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:17.773765 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e767ede8-34b1-47b5-9313-1cee30e02a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:17.883804 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 29/63] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:17.920334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1580a-7871-4cbd-8e96-a6a27976078c None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:17.969941 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:18.095925 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1580a-7871-4cbd-8e96-a6a27976078c None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:18.148771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb582c-aa65-4a4a-9e79-180dd03bfbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:18.149962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb582c-aa65-4a4a-9e79-180dd03bfbe1 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:18.150483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb582c-aa65-4a4a-9e79-180dd03bfbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:18.150973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb582c-aa65-4a4a-9e79-180dd03bfbe1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:18.175329 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb582c-aa65-4a4a-9e79-180dd03bfbe1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:18.183567 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-cfdb582c-aa65-4a4a-9e79-180dd03bfbe1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:40:18.183567 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:18.183567 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:18.183567 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:18.183567 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:18.183567 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:18.183567 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:18.183567 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:18.183567 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:18.183567 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:18.183567 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:18.183567 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:18.183567 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:18.183567 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:18.183567 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:18.183567 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:18.184582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:18.184582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:18.184582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:18.184582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:18.184582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:18.184582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:18.184582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:18.184582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:18.184582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:18.184582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:18.184582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:18.184582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:40:18.184582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:18.184582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:18.184582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:18.184582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:18.185353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:18.185353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:18.185353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:18.185353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:18.185353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:18.185353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:18.185353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:18.185353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:18.185353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:18.185353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:18.185353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:40:18.185353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:40:18.187826 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 35/64] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:18.194229 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767ede8-34b1-47b5-9313-1cee30e02a62 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:18.268161 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2855f3d-01c5-4478-a48a-bcbe63f3f11e None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:18.268777 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2855f3d-01c5-4478-a48a-bcbe63f3f11e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:18.269243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2855f3d-01c5-4478-a48a-bcbe63f3f11e None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:18.274129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 36/65] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:18.294189 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c8973-769c-46bb-b44a-d1ec62b4d5f3 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:18.294847 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c8973-769c-46bb-b44a-d1ec62b4d5f3 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:18.295332 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c8973-769c-46bb-b44a-d1ec62b4d5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:18.326031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-229c8973-769c-46bb-b44a-d1ec62b4d5f3 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:18.327510 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-229c8973-769c-46bb-b44a-d1ec62b4d5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:18.338822 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:18.357522 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767ede8-34b1-47b5-9313-1cee30e02a62 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:18.395125 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfec3ea-c062-4868-871b-12cc62f50c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:18.395955 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfec3ea-c062-4868-871b-12cc62f50c36 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:18.396316 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfec3ea-c062-4868-871b-12cc62f50c36 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:18.396666 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfec3ea-c062-4868-871b-12cc62f50c36 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:18.397519 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfec3ea-c062-4868-871b-12cc62f50c36 None admin] RBAC: Adding query filter params (name=member) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:18.397893 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfec3ea-c062-4868-871b-12cc62f50c36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:18.399506 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfec3ea-c062-4868-871b-12cc62f50c36 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:18.410697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 30/66] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:40:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:18.412537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0fb9b-53b3-4061-b766-8087637c2ec0 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:18.413044 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0fb9b-53b3-4061-b766-8087637c2ec0 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:18.413645 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0fb9b-53b3-4061-b766-8087637c2ec0 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:18.416038 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 31/67] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:18.421406 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0fb9b-53b3-4061-b766-8087637c2ec0 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:18.439456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 37/68] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:40:18.441366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a86e0-4361-42c2-b3ec-d4b1df94a9bb None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:18.441670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a86e0-4361-42c2-b3ec-d4b1df94a9bb None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:18.441934 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a86e0-4361-42c2-b3ec-d4b1df94a9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:18.465120 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-659a86e0-4361-42c2-b3ec-d4b1df94a9bb None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:18.466361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-659a86e0-4361-42c2-b3ec-d4b1df94a9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:18.480232 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:18.558160 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 38/69] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:18.558160 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a02ab-14e3-48fb-a78a-d9d9f5a47a45 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:18.558160 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a02ab-14e3-48fb-a78a-d9d9f5a47a45 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:18.558160 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a02ab-14e3-48fb-a78a-d9d9f5a47a45 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:18.558160 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 39/70] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:18.558160 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a02ab-14e3-48fb-a78a-d9d9f5a47a45 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:18.560227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0fb9b-53b3-4061-b766-8087637c2ec0 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:18.601695 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ab062-9bc2-471d-ad0e-fa2bb79409c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:18.602873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ab062-9bc2-471d-ad0e-fa2bb79409c7 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:18.603164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ab062-9bc2-471d-ad0e-fa2bb79409c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:18.603445 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ab062-9bc2-471d-ad0e-fa2bb79409c7 None admin] PATH_INFO: `/v3/users/74e87dd6b03d4b6ca41070df6e3f2f11` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:18.622449 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ab062-9bc2-471d-ad0e-fa2bb79409c7 None admin] RBAC: Authorizing `identity:get_user(user_id=74e87dd6b03d4b6ca41070df6e3f2f11)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:18.623864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ab062-9bc2-471d-ad0e-fa2bb79409c7 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:18.630598 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 32/71] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:18 2022] GET /identity/v3/users/74e87dd6b03d4b6ca41070df6e3f2f11 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:18.630598 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c98a7b-3a2b-494c-b097-2259d235d1e6 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:18.630598 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c98a7b-3a2b-494c-b097-2259d235d1e6 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:18.630598 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c98a7b-3a2b-494c-b097-2259d235d1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:18.636256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:18.651965 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-82c98a7b-3a2b-494c-b097-2259d235d1e6 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:18.653241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-82c98a7b-3a2b-494c-b097-2259d235d1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:18.690467 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a02ab-14e3-48fb-a78a-d9d9f5a47a45 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:18.726692 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 33/72] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:18.727032 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8425dd2-c531-4d71-b3b0-5144bd0b0553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:18.727149 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c98a7b-3a2b-494c-b097-2259d235d1e6 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:18.727227 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8425dd2-c531-4d71-b3b0-5144bd0b0553 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:18.727476 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8425dd2-c531-4d71-b3b0-5144bd0b0553 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:18.727702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8425dd2-c531-4d71-b3b0-5144bd0b0553 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:18.738816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8425dd2-c531-4d71-b3b0-5144bd0b0553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:18.748087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-d8425dd2-c531-4d71-b3b0-5144bd0b0553 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:40:18.748087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:18.748087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:18.748087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:18.748087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:18.748087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:18.748087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:18.748087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:18.748087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:18.748087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:18.748087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:18.748087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:18.748087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:18.748087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:18.748087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:18.748087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:18.748983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:18.748983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:18.748983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:18.748983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:18.748983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:18.748983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:18.748983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:18.748983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:18.748983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:18.748983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:18.748983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:18.748983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:40:18.748983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:18.748983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:18.748983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:18.748983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:18.749752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:18.749752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:18.749752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:18.749752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:18.749752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:18.749752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:18.749752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:18.749752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:18.749752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:18.749752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:18.749752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:40:18.749752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:40:18.755209 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 40/73] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:40:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:18.755209 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8425dd2-c531-4d71-b3b0-5144bd0b0553 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:18.760307 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:18.849850 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c98a7b-3a2b-494c-b097-2259d235d1e6 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:18.883986 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8425dd2-c531-4d71-b3b0-5144bd0b0553 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:18.888850 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f589c-1aa8-43b7-9121-adb9d5ab8ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:18.889672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f589c-1aa8-43b7-9121-adb9d5ab8ff4 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:18.889962 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f589c-1aa8-43b7-9121-adb9d5ab8ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:18.890281 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f589c-1aa8-43b7-9121-adb9d5ab8ff4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:18.892659 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:18.910976 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414f589c-1aa8-43b7-9121-adb9d5ab8ff4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:18.920786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-414f589c-1aa8-43b7-9121-adb9d5ab8ff4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:40:18.920786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:18.920786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:18.920786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:18.920786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:18.920786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:18.920786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:18.920786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:18.920786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:18.920786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:18.920786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:18.920786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:18.920786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:18.920786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:18.920786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:18.920786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:18.921717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:18.921717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:18.921717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:18.921717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:18.921717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:18.921717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:18.921717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:18.921717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:18.921717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:18.921717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:18.921717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:18.921717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:40:18.921717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:18.921717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:18.921717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:18.921717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:18.922531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:18.922531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:18.922531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:18.922531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:18.922531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:18.922531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:18.922531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:18.922531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:18.922531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:18.922531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:18.922531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:40:18.922531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:40:18.922531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 34/74] 10.222.0.78 () {56 vars in 1114 bytes} [Wed May 25 14:40:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:18.923552 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035a429-bf52-48c5-bd2b-53ce219d3ce2 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:18.923749 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035a429-bf52-48c5-bd2b-53ce219d3ce2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:18.924042 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035a429-bf52-48c5-bd2b-53ce219d3ce2 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:18.926058 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 35/75] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:18.926889 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035a429-bf52-48c5-bd2b-53ce219d3ce2 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:18.951352 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3fd30f-5eee-4dbc-80f6-d1e508270679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:18.952480 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3fd30f-5eee-4dbc-80f6-d1e508270679 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:18.952979 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3fd30f-5eee-4dbc-80f6-d1e508270679 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:18.953464 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3fd30f-5eee-4dbc-80f6-d1e508270679 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:19.026551 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3fd30f-5eee-4dbc-80f6-d1e508270679 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:19.028039 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3fd30f-5eee-4dbc-80f6-d1e508270679 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:19.030989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:19.034315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035a429-bf52-48c5-bd2b-53ce219d3ce2 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:19.070294 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70990dc1-34dc-4dab-b824-15bd6678cacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:19.070964 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70990dc1-34dc-4dab-b824-15bd6678cacb None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:19.071258 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70990dc1-34dc-4dab-b824-15bd6678cacb None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:19.071526 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70990dc1-34dc-4dab-b824-15bd6678cacb None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:19.072279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70990dc1-34dc-4dab-b824-15bd6678cacb None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:19.072587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70990dc1-34dc-4dab-b824-15bd6678cacb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:19.073992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70990dc1-34dc-4dab-b824-15bd6678cacb None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:19.082857 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 36/76] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:19.083703 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70990dc1-34dc-4dab-b824-15bd6678cacb None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:19.139831 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 41/77] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:40:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:40:19.140597 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3fd30f-5eee-4dbc-80f6-d1e508270679 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:19.146120 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:19.197713 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70990dc1-34dc-4dab-b824-15bd6678cacb None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:19.235058 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc6333-7471-4473-b1c4-dff0ac71790e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:19.235949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc6333-7471-4473-b1c4-dff0ac71790e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:19.236232 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc6333-7471-4473-b1c4-dff0ac71790e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:19.236515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc6333-7471-4473-b1c4-dff0ac71790e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:19.238382 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc6333-7471-4473-b1c4-dff0ac71790e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:19.239886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc6333-7471-4473-b1c4-dff0ac71790e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:19.241532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 37/78] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:40:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:19.242753 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a30f7-8e1c-4a7c-9894-5aab275587c5 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:19.243036 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a30f7-8e1c-4a7c-9894-5aab275587c5 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:19.243319 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a30f7-8e1c-4a7c-9894-5aab275587c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:19.253228 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3fd30f-5eee-4dbc-80f6-d1e508270679 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:19.255614 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:19.260262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-f97a30f7-8e1c-4a7c-9894-5aab275587c5 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:19.261481 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f97a30f7-8e1c-4a7c-9894-5aab275587c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:19.296563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b621f-0fd4-4a87-9706-da94212d5de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:19.297235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b621f-0fd4-4a87-9706-da94212d5de7 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:19.297517 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b621f-0fd4-4a87-9706-da94212d5de7 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:19.297797 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b621f-0fd4-4a87-9706-da94212d5de7 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:19.298590 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b621f-0fd4-4a87-9706-da94212d5de7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:19.298898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b621f-0fd4-4a87-9706-da94212d5de7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:19.300791 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b621f-0fd4-4a87-9706-da94212d5de7 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:19.309338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 42/79] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:40:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:19.310222 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b621f-0fd4-4a87-9706-da94212d5de7 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:19.338735 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 38/80] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:19.339614 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a30f7-8e1c-4a7c-9894-5aab275587c5 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:19.419772 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:19.435547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b621f-0fd4-4a87-9706-da94212d5de7 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:19.471638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a30f7-8e1c-4a7c-9894-5aab275587c5 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:19.474527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baad5dd-df7f-4914-9797-207f7d9d5db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:19.475457 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baad5dd-df7f-4914-9797-207f7d9d5db1 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:19.475764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baad5dd-df7f-4914-9797-207f7d9d5db1 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:19.476046 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baad5dd-df7f-4914-9797-207f7d9d5db1 None admin] PATH_INFO: `/v3/users/f27fbbbc33b24630948a570ccae5fad0` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:19.477494 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baad5dd-df7f-4914-9797-207f7d9d5db1 None admin] RBAC: Authorizing `identity:get_user(user_id=f27fbbbc33b24630948a570ccae5fad0)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:19.479241 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baad5dd-df7f-4914-9797-207f7d9d5db1 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:19.481053 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 43/81] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:19 2022] GET /identity/v3/users/f27fbbbc33b24630948a570ccae5fad0 => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:19.481861 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baad5dd-df7f-4914-9797-207f7d9d5db1 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:19.514084 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09034a-92bb-4b3e-b1c1-5df9fcb3f993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:19.514952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09034a-92bb-4b3e-b1c1-5df9fcb3f993 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:19.515238 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09034a-92bb-4b3e-b1c1-5df9fcb3f993 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:19.515515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09034a-92bb-4b3e-b1c1-5df9fcb3f993 None admin] PATH_INFO: `/v3/projects/9799d178c2a042baa9963766f953c6a6` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:19.516732 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b09034a-92bb-4b3e-b1c1-5df9fcb3f993 None admin] RBAC: Authorizing `identity:get_project(project_id=9799d178c2a042baa9963766f953c6a6)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:19.518254 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b09034a-92bb-4b3e-b1c1-5df9fcb3f993 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:19.520509 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 39/82] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:19 2022] GET /identity/v3/projects/9799d178c2a042baa9963766f953c6a6 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:19.521297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09034a-92bb-4b3e-b1c1-5df9fcb3f993 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:19.522082 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:19.595001 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baad5dd-df7f-4914-9797-207f7d9d5db1 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:19.625741 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09034a-92bb-4b3e-b1c1-5df9fcb3f993 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:19.639652 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97150035-b31e-4dba-abb9-5630aae06886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:19.640487 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97150035-b31e-4dba-abb9-5630aae06886 None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:19.640769 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97150035-b31e-4dba-abb9-5630aae06886 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:19.641050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97150035-b31e-4dba-abb9-5630aae06886 None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:19.641785 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97150035-b31e-4dba-abb9-5630aae06886 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:19.643309 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97150035-b31e-4dba-abb9-5630aae06886 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:19.651588 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:19.666980 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcfde0-4fba-445e-a777-bbac4109ed31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:19.668082 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcfde0-4fba-445e-a777-bbac4109ed31 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:19.668376 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcfde0-4fba-445e-a777-bbac4109ed31 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:19.668762 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcfde0-4fba-445e-a777-bbac4109ed31 None admin] PATH_INFO: `/v3/users/74e87dd6b03d4b6ca41070df6e3f2f11` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:19.670293 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dcfde0-4fba-445e-a777-bbac4109ed31 None admin] RBAC: Authorizing `identity:get_user(user_id=74e87dd6b03d4b6ca41070df6e3f2f11)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:19.671727 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dcfde0-4fba-445e-a777-bbac4109ed31 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:19.673438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 40/83] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:19 2022] GET /identity/v3/users/74e87dd6b03d4b6ca41070df6e3f2f11 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:19.680461 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 44/84] 10.222.0.78 () {60 vars in 1132 bytes} [Wed May 25 14:40:19 2022] POST /identity/v3/users => generated 289 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:19.680660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcfde0-4fba-445e-a777-bbac4109ed31 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:19.680749 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97150035-b31e-4dba-abb9-5630aae06886 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:19.801393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:19.813456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcfde0-4fba-445e-a777-bbac4109ed31 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:19.817746 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97150035-b31e-4dba-abb9-5630aae06886 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:19.858089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d44afac-0dab-4323-b897-4b30368ed1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:19.858792 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44afac-0dab-4323-b897-4b30368ed1d2 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:19.859021 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c17e99-58d2-4d66-9382-3cbd9d997a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:19.859485 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44afac-0dab-4323-b897-4b30368ed1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:19.859485 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44afac-0dab-4323-b897-4b30368ed1d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:19.860123 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c17e99-58d2-4d66-9382-3cbd9d997a03 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:19.860399 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c17e99-58d2-4d66-9382-3cbd9d997a03 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:19.860676 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c17e99-58d2-4d66-9382-3cbd9d997a03 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:19.878988 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c17e99-58d2-4d66-9382-3cbd9d997a03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:19.887470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-b4c17e99-58d2-4d66-9382-3cbd9d997a03 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:40:19.887470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:19.887470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:19.887470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:19.887470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:19.887470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:19.887470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:19.887470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:19.887470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:19.887470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:19.887470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:19.887470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:19.887470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:19.887470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:19.887470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:19.887470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:19.888393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:19.888393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:19.888393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:19.888393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:19.888393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:19.888393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:19.888393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:19.888393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:19.888393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:19.888393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:19.888393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:19.888393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:40:19.888393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:19.888393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:19.888393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:19.888393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:19.889138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:19.889138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:19.889138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:19.889138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:19.889138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:19.889138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:19.889138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:19.889138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:19.889138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:19.889138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:19.889138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:40:19.889138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:40:19.889138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 41/85] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:19.889751 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c17e99-58d2-4d66-9382-3cbd9d997a03 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:19.975755 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d44afac-0dab-4323-b897-4b30368ed1d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:19.978807 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d44afac-0dab-4323-b897-4b30368ed1d2 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:19.983544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:20.065039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c17e99-58d2-4d66-9382-3cbd9d997a03 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:20.116741 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa26e9-260d-48f8-bc1f-9d7345a1b4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:20.117469 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa26e9-260d-48f8-bc1f-9d7345a1b4aa None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:20.118131 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa26e9-260d-48f8-bc1f-9d7345a1b4aa None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:20.118631 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa26e9-260d-48f8-bc1f-9d7345a1b4aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:20.119517 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa26e9-260d-48f8-bc1f-9d7345a1b4aa None admin] RBAC: Adding query filter params (role.id=a3da3c62738d4790b9dac512b1568b52, scope.project.id=9799d178c2a042baa9963766f953c6a6, user.id=74e87dd6b03d4b6ca41070df6e3f2f11) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:20.119839 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa26e9-260d-48f8-bc1f-9d7345a1b4aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:20.121575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa26e9-260d-48f8-bc1f-9d7345a1b4aa None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:20.130087 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 42/86] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:40:19 2022] GET /identity/v3/role_assignments?user.id=74e87dd6b03d4b6ca41070df6e3f2f11&scope.project.id=9799d178c2a042baa9963766f953c6a6&role.id=a3da3c62738d4790b9dac512b1568b52 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:20.131036 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa26e9-260d-48f8-bc1f-9d7345a1b4aa None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:20.186401 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:20.202988 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 45/87] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:40:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:40:20.203916 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d44afac-0dab-4323-b897-4b30368ed1d2 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:20.356302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d44afac-0dab-4323-b897-4b30368ed1d2 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:20.403676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b8083-676a-4e23-b051-1b79ea96968b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:20.404759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b8083-676a-4e23-b051-1b79ea96968b None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:20.405243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b8083-676a-4e23-b051-1b79ea96968b None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:20.405726 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b8083-676a-4e23-b051-1b79ea96968b None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:20.407170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b8083-676a-4e23-b051-1b79ea96968b None admin] RBAC: Adding query filter params (name=member) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:20.407737 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b8083-676a-4e23-b051-1b79ea96968b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:20.410355 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b8083-676a-4e23-b051-1b79ea96968b None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:20.415050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa26e9-260d-48f8-bc1f-9d7345a1b4aa None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:20.426990 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 46/88] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:40:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:20.428331 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b8083-676a-4e23-b051-1b79ea96968b None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:20.433358 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:20.458915 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52592673-9ee7-4881-bded-c2afedee08a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:20.459752 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52592673-9ee7-4881-bded-c2afedee08a0 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:20.460152 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52592673-9ee7-4881-bded-c2afedee08a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:20.460534 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52592673-9ee7-4881-bded-c2afedee08a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:20.519481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:20.577363 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b8083-676a-4e23-b051-1b79ea96968b None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:20.598704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52592673-9ee7-4881-bded-c2afedee08a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:20.600231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52592673-9ee7-4881-bded-c2afedee08a0 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:20.625784 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3abb0-13f3-4672-9cd6-ed17e2659bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:20.626890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3abb0-13f3-4672-9cd6-ed17e2659bd5 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:20.627177 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3abb0-13f3-4672-9cd6-ed17e2659bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:20.627453 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3abb0-13f3-4672-9cd6-ed17e2659bd5 None admin] PATH_INFO: `/v3/projects/9799d178c2a042baa9963766f953c6a6` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:20.629947 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3abb0-13f3-4672-9cd6-ed17e2659bd5 None admin] RBAC: Authorizing `identity:get_project(project_id=9799d178c2a042baa9963766f953c6a6)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:20.631756 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3abb0-13f3-4672-9cd6-ed17e2659bd5 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:20.633626 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 47/89] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:20 2022] GET /identity/v3/projects/9799d178c2a042baa9963766f953c6a6 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:20.640010 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3abb0-13f3-4672-9cd6-ed17e2659bd5 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:20.814435 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3abb0-13f3-4672-9cd6-ed17e2659bd5 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:20.858463 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:20.904381 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f61da5-995d-4320-8ce0-5040ed1284ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:20.905509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f61da5-995d-4320-8ce0-5040ed1284ef None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:20.910084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f61da5-995d-4320-8ce0-5040ed1284ef None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:20.910084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f61da5-995d-4320-8ce0-5040ed1284ef None admin] PATH_INFO: `/v3/users/74e87dd6b03d4b6ca41070df6e3f2f11` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:20.911757 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f61da5-995d-4320-8ce0-5040ed1284ef None admin] RBAC: Authorizing `identity:get_user(user_id=74e87dd6b03d4b6ca41070df6e3f2f11)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:20.915423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f61da5-995d-4320-8ce0-5040ed1284ef None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:20.923344 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 43/91] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:40:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 793 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:40:20.924813 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52592673-9ee7-4881-bded-c2afedee08a0 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:20.925686 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 48/91] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:20 2022] GET /identity/v3/users/74e87dd6b03d4b6ca41070df6e3f2f11 => generated 275 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:20.927348 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f61da5-995d-4320-8ce0-5040ed1284ef None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:20.960388 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:21.136476 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52592673-9ee7-4881-bded-c2afedee08a0 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:21.182442 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa82b41-ffaa-4ec7-a304-9b8074e85c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:21.183055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa82b41-ffaa-4ec7-a304-9b8074e85c55 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:21.183343 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa82b41-ffaa-4ec7-a304-9b8074e85c55 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:21.183626 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa82b41-ffaa-4ec7-a304-9b8074e85c55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:21.184493 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa82b41-ffaa-4ec7-a304-9b8074e85c55 None admin] RBAC: Adding query filter params (role.id=e566b37cd92f467198b5dc40f183effc, scope.project.id=9799d178c2a042baa9963766f953c6a6, user.id=f27fbbbc33b24630948a570ccae5fad0) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:21.184809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa82b41-ffaa-4ec7-a304-9b8074e85c55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:21.185512 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f61da5-995d-4320-8ce0-5040ed1284ef None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:21.186380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa82b41-ffaa-4ec7-a304-9b8074e85c55 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:21.194887 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 44/92] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:40:20 2022] GET /identity/v3/role_assignments?user.id=f27fbbbc33b24630948a570ccae5fad0&scope.project.id=9799d178c2a042baa9963766f953c6a6&role.id=e566b37cd92f467198b5dc40f183effc => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:21.196958 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa82b41-ffaa-4ec7-a304-9b8074e85c55 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:21.362465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e4413a-a8c8-49b7-abd9-c01d10d1d830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:21.364277 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4413a-a8c8-49b7-abd9-c01d10d1d830 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:21.364703 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4413a-a8c8-49b7-abd9-c01d10d1d830 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:21.365117 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4413a-a8c8-49b7-abd9-c01d10d1d830 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:21.402434 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:21.452128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa82b41-ffaa-4ec7-a304-9b8074e85c55 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:21.490356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:21.519722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e4413a-a8c8-49b7-abd9-c01d10d1d830 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:21.521417 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e4413a-a8c8-49b7-abd9-c01d10d1d830 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:21.527006 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb435e6-8d7f-4ea1-8bbe-a21ddc50d31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:21.527927 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb435e6-8d7f-4ea1-8bbe-a21ddc50d31d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:21.528214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb435e6-8d7f-4ea1-8bbe-a21ddc50d31d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:21.528612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb435e6-8d7f-4ea1-8bbe-a21ddc50d31d None admin] PATH_INFO: `/v3/projects/9799d178c2a042baa9963766f953c6a6` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:21.530788 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb435e6-8d7f-4ea1-8bbe-a21ddc50d31d None admin] RBAC: Authorizing `identity:get_project(project_id=9799d178c2a042baa9963766f953c6a6)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:21.532303 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb435e6-8d7f-4ea1-8bbe-a21ddc50d31d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:21.538938 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 45/93] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:21 2022] GET /identity/v3/projects/9799d178c2a042baa9963766f953c6a6 => generated 298 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:21.546809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb435e6-8d7f-4ea1-8bbe-a21ddc50d31d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:21.736477 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 49/94] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:40:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 810 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:40:21.743988 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e4413a-a8c8-49b7-abd9-c01d10d1d830 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:21.749357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb435e6-8d7f-4ea1-8bbe-a21ddc50d31d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:21.818799 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729823be-ec93-4b6b-acc8-d16c1dfe223e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:21.819542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729823be-ec93-4b6b-acc8-d16c1dfe223e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:21.819857 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729823be-ec93-4b6b-acc8-d16c1dfe223e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:21.820142 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729823be-ec93-4b6b-acc8-d16c1dfe223e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:21.821023 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729823be-ec93-4b6b-acc8-d16c1dfe223e None admin] RBAC: Adding query filter params (role.id=d451467ccbbf4149abbfc9b0f76b9339, scope.project.id=9799d178c2a042baa9963766f953c6a6, user.id=74e87dd6b03d4b6ca41070df6e3f2f11) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:21.821345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729823be-ec93-4b6b-acc8-d16c1dfe223e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:21.824063 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729823be-ec93-4b6b-acc8-d16c1dfe223e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:21.835713 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 46/95] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:40:21 2022] GET /identity/v3/role_assignments?user.id=74e87dd6b03d4b6ca41070df6e3f2f11&scope.project.id=9799d178c2a042baa9963766f953c6a6&role.id=d451467ccbbf4149abbfc9b0f76b9339 => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:21.904718 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:21.921365 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e4413a-a8c8-49b7-abd9-c01d10d1d830 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:21.976937 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053bc0a1-72e2-495a-a4d1-4205da5f0626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:21.977822 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bc0a1-72e2-495a-a4d1-4205da5f0626 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:21.978131 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bc0a1-72e2-495a-a4d1-4205da5f0626 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:21.978517 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bc0a1-72e2-495a-a4d1-4205da5f0626 None admin] PATH_INFO: `/v3/projects/39ae31a66cc64d0095d6f865e887f407` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:21.980104 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053bc0a1-72e2-495a-a4d1-4205da5f0626 None admin] RBAC: Authorizing `identity:get_project(project_id=39ae31a66cc64d0095d6f865e887f407)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:21.981649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053bc0a1-72e2-495a-a4d1-4205da5f0626 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:21.983766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 50/96] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:21 2022] GET /identity/v3/projects/39ae31a66cc64d0095d6f865e887f407 => generated 312 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:21.993611 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729823be-ec93-4b6b-acc8-d16c1dfe223e None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:22.019872 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:22.145734 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729823be-ec93-4b6b-acc8-d16c1dfe223e None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:22.194956 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ade56-dbd6-47d1-978b-7ff1b19ab1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:22.195665 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ade56-dbd6-47d1-978b-7ff1b19ab1d3 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:22.195956 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ade56-dbd6-47d1-978b-7ff1b19ab1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:22.196242 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ade56-dbd6-47d1-978b-7ff1b19ab1d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:22.197125 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ade56-dbd6-47d1-978b-7ff1b19ab1d3 None admin] RBAC: Adding query filter params (role.id=a3da3c62738d4790b9dac512b1568b52, scope.project.id=39ae31a66cc64d0095d6f865e887f407, user.id=74e87dd6b03d4b6ca41070df6e3f2f11) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:22.197449 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ade56-dbd6-47d1-978b-7ff1b19ab1d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:22.199031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ade56-dbd6-47d1-978b-7ff1b19ab1d3 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:22.211236 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 47/97] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:40:21 2022] GET /identity/v3/role_assignments?user.id=74e87dd6b03d4b6ca41070df6e3f2f11&scope.project.id=39ae31a66cc64d0095d6f865e887f407&role.id=a3da3c62738d4790b9dac512b1568b52 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:23.059578 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103e305-71e3-4b85-a4a8-f77c91c0c358 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:23.060651 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103e305-71e3-4b85-a4a8-f77c91c0c358 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:23.061157 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103e305-71e3-4b85-a4a8-f77c91c0c358 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:23.062987 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 51/98] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:40:23.069505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebf2a4-79bb-4bb8-9609-19713cce2db2 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:23.070583 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebf2a4-79bb-4bb8-9609-19713cce2db2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:23.071107 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebf2a4-79bb-4bb8-9609-19713cce2db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:23.093672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-61ebf2a4-79bb-4bb8-9609-19713cce2db2 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:23.095269 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-61ebf2a4-79bb-4bb8-9609-19713cce2db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:23.192092 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 48/99] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:23.235591 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103e305-71e3-4b85-a4a8-f77c91c0c358 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:23.246309 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f30c92-81da-4391-928b-18de2ca6b5d6 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:23.248970 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f30c92-81da-4391-928b-18de2ca6b5d6 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:23.249546 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f30c92-81da-4391-928b-18de2ca6b5d6 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:23.255667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 49/100] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:23 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:23.262733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19829cfa-62a8-4f4f-9630-e00a0c044360 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:23.263330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19829cfa-62a8-4f4f-9630-e00a0c044360 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:23.263982 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19829cfa-62a8-4f4f-9630-e00a0c044360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:23.279396 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:23.302574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-19829cfa-62a8-4f4f-9630-e00a0c044360 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:23.304159 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-19829cfa-62a8-4f4f-9630-e00a0c044360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:23.448517 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 50/101] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:40:23.450921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07009c7-ac9f-44fd-ac6a-7dca5517ec07 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:23.451914 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07009c7-ac9f-44fd-ac6a-7dca5517ec07 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:23.452965 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07009c7-ac9f-44fd-ac6a-7dca5517ec07 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:23.457233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 51/102] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:23.465400 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103e305-71e3-4b85-a4a8-f77c91c0c358 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:23.471903 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9f1d8-1d7a-48c1-b1e1-46231365bd92 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:23.472603 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9f1d8-1d7a-48c1-b1e1-46231365bd92 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:23.473347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9f1d8-1d7a-48c1-b1e1-46231365bd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:23.516802 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-76b9f1d8-1d7a-48c1-b1e1-46231365bd92 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:23.520844 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-76b9f1d8-1d7a-48c1-b1e1-46231365bd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:23.560419 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84afc2-55b3-4603-832a-2d97c081f748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:23.561348 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84afc2-55b3-4603-832a-2d97c081f748 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:23.561713 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84afc2-55b3-4603-832a-2d97c081f748 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:23.561995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84afc2-55b3-4603-832a-2d97c081f748 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:23.567069 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b84afc2-55b3-4603-832a-2d97c081f748 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:23.568599 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b84afc2-55b3-4603-832a-2d97c081f748 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:23.571792 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 52/103] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:40:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:23.572647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84afc2-55b3-4603-832a-2d97c081f748 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:23.633863 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 52/104] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:23.635040 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9f1d8-1d7a-48c1-b1e1-46231365bd92 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:23.655799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:23.719120 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:23.770735 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9f1d8-1d7a-48c1-b1e1-46231365bd92 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:23.820942 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec29109-8d69-4fce-adda-2abf2a456da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:23.821930 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec29109-8d69-4fce-adda-2abf2a456da0 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:23.822451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec29109-8d69-4fce-adda-2abf2a456da0 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:23.822924 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec29109-8d69-4fce-adda-2abf2a456da0 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:23.823829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec29109-8d69-4fce-adda-2abf2a456da0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:23.825639 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec29109-8d69-4fce-adda-2abf2a456da0 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:23.863850 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 53/105] 10.222.0.78 () {60 vars in 1141 bytes} [Wed May 25 14:40:23 2022] POST /identity/v3/projects => generated 302 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:23.867389 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec29109-8d69-4fce-adda-2abf2a456da0 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:23.930765 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84afc2-55b3-4603-832a-2d97c081f748 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:24.009007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec29109-8d69-4fce-adda-2abf2a456da0 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:24.011289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006e56d-aada-4120-8dc9-b010abdfe8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:24.012463 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006e56d-aada-4120-8dc9-b010abdfe8c5 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:24.012911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006e56d-aada-4120-8dc9-b010abdfe8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:24.013353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006e56d-aada-4120-8dc9-b010abdfe8c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:24.030858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:24.047139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b006e56d-aada-4120-8dc9-b010abdfe8c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:24.055603 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b0b41-8296-48bd-8285-c85122ed192c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:24.056931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b0b41-8296-48bd-8285-c85122ed192c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:24.057436 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b0b41-8296-48bd-8285-c85122ed192c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:24.057958 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b0b41-8296-48bd-8285-c85122ed192c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:24.063603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-b006e56d-aada-4120-8dc9-b010abdfe8c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:40:24.063603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:24.063603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:24.063603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:24.063603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:24.063603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:24.063603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:24.063603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:24.063603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:24.063603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:24.063603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:24.063603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:24.063603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:24.063603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:24.063603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:24.063603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:24.064852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:24.064852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:24.064852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:24.064852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:24.064852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:24.064852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:24.064852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:24.064852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:24.064852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:24.064852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:24.064852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:24.064852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:40:24.064852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:24.064852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:24.064852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:24.064852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:24.065645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:24.065645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:24.065645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:24.065645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:24.065645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:24.065645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:24.065645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:24.065645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:24.065645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:24.065645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:24.065645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:40:24.065645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:40:24.065645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 53/106] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 493 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:24.075093 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006e56d-aada-4120-8dc9-b010abdfe8c5 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:24.076573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b0b41-8296-48bd-8285-c85122ed192c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:24.086250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-994b0b41-8296-48bd-8285-c85122ed192c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:40:24.086250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:24.086250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:24.086250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:24.086250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:24.086250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:24.086250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:24.086250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:24.086250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:24.086250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:24.086250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:24.086250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:24.086250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:24.086250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:24.086250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:24.086250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:24.087188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:24.087188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:24.087188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:24.087188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:24.087188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:24.087188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:24.087188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:24.087188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:24.087188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:24.087188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:24.087188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:24.087188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:40:24.087188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:24.087188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:24.087188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:24.087188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:24.087955 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:24.087955 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:24.087955 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:24.087955 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:24.087955 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:24.087955 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:24.087955 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:24.087955 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:24.087955 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:24.087955 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:24.087955 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:40:24.087955 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:40:24.090907 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 54/107] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:24.099268 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b0b41-8296-48bd-8285-c85122ed192c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:24.309272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:24.329387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006e56d-aada-4120-8dc9-b010abdfe8c5 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:24.367737 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b0b41-8296-48bd-8285-c85122ed192c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:24.374230 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcdc434-7b4e-457f-b336-e181f8de2d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:24.374996 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdc434-7b4e-457f-b336-e181f8de2d1b None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:24.375489 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdc434-7b4e-457f-b336-e181f8de2d1b None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:24.375773 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdc434-7b4e-457f-b336-e181f8de2d1b None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:24.376988 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcdc434-7b4e-457f-b336-e181f8de2d1b None admin] RBAC: Adding query filter params (name=member) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:24.377319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcdc434-7b4e-457f-b336-e181f8de2d1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:24.379733 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcdc434-7b4e-457f-b336-e181f8de2d1b None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:24.391002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 54/108] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:40:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:24.397643 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcdc434-7b4e-457f-b336-e181f8de2d1b None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:24.483367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1314d9-63f2-4fbb-bbe9-cabb9d4f9f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:24.484064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1314d9-63f2-4fbb-bbe9-cabb9d4f9f44 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:24.484347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1314d9-63f2-4fbb-bbe9-cabb9d4f9f44 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:24.484643 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1314d9-63f2-4fbb-bbe9-cabb9d4f9f44 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:24.485383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1314d9-63f2-4fbb-bbe9-cabb9d4f9f44 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:24.485691 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1314d9-63f2-4fbb-bbe9-cabb9d4f9f44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:24.487834 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1314d9-63f2-4fbb-bbe9-cabb9d4f9f44 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:24.503431 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 55/109] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:40:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:24.508836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1314d9-63f2-4fbb-bbe9-cabb9d4f9f44 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:24.532383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcdc434-7b4e-457f-b336-e181f8de2d1b None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:24.574845 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac8b35-10eb-478b-b794-b40f64749394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:24.575651 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac8b35-10eb-478b-b794-b40f64749394 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:24.575937 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac8b35-10eb-478b-b794-b40f64749394 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:24.576211 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac8b35-10eb-478b-b794-b40f64749394 None admin] PATH_INFO: `/v3/users/74e87dd6b03d4b6ca41070df6e3f2f11` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:24.577624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac8b35-10eb-478b-b794-b40f64749394 None admin] RBAC: Authorizing `identity:get_user(user_id=74e87dd6b03d4b6ca41070df6e3f2f11)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:24.579265 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac8b35-10eb-478b-b794-b40f64749394 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:24.580996 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 55/110] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:24 2022] GET /identity/v3/users/74e87dd6b03d4b6ca41070df6e3f2f11 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:24.587095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac8b35-10eb-478b-b794-b40f64749394 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:24.588924 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:24.608170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:24.696212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1314d9-63f2-4fbb-bbe9-cabb9d4f9f44 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:24.761533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ad936-3332-4455-8794-31ac55fe0aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:24.763019 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ad936-3332-4455-8794-31ac55fe0aaf None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:24.763422 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ad936-3332-4455-8794-31ac55fe0aaf None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:24.763897 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ad936-3332-4455-8794-31ac55fe0aaf None admin] PATH_INFO: `/v3/users/b3413b1ed0e54a73834c597e8e23a440` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:24.810338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ad936-3332-4455-8794-31ac55fe0aaf None admin] RBAC: Authorizing `identity:get_user(user_id=b3413b1ed0e54a73834c597e8e23a440)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:24.812688 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ad936-3332-4455-8794-31ac55fe0aaf None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:24.814945 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 56/111] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:24 2022] GET /identity/v3/users/b3413b1ed0e54a73834c597e8e23a440 => generated 289 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:24.822292 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac8b35-10eb-478b-b794-b40f64749394 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:24.826603 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c269c1d-205c-48fa-bd6b-3d3e001acf5c None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:24.827011 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c269c1d-205c-48fa-bd6b-3d3e001acf5c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:24.827472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c269c1d-205c-48fa-bd6b-3d3e001acf5c None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:24.830556 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 57/112] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:24 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:24.834345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c269c1d-205c-48fa-bd6b-3d3e001acf5c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:24.920390 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e0c76-2ff1-434c-b464-a85982cb7007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:24.921303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e0c76-2ff1-434c-b464-a85982cb7007 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:24.921589 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e0c76-2ff1-434c-b464-a85982cb7007 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:24.921856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e0c76-2ff1-434c-b464-a85982cb7007 None admin] PATH_INFO: `/v3/projects/9799d178c2a042baa9963766f953c6a6` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:24.926982 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e0c76-2ff1-434c-b464-a85982cb7007 None admin] RBAC: Authorizing `identity:get_project(project_id=9799d178c2a042baa9963766f953c6a6)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:24.928591 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e0c76-2ff1-434c-b464-a85982cb7007 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:24.930730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 56/113] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:24 2022] GET /identity/v3/projects/9799d178c2a042baa9963766f953c6a6 => generated 298 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:24.932092 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92d260-19cc-402b-8613-6cd9882a9424 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:24.932475 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92d260-19cc-402b-8613-6cd9882a9424 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:24.932763 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92d260-19cc-402b-8613-6cd9882a9424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:24.958630 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-be92d260-19cc-402b-8613-6cd9882a9424 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:24.959953 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-be92d260-19cc-402b-8613-6cd9882a9424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:25.039820 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:25.048901 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c269c1d-205c-48fa-bd6b-3d3e001acf5c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:25.052511 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 57/114] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:25.053764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92d260-19cc-402b-8613-6cd9882a9424 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:25.070476 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:25.095965 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da7c0f2-7595-42b4-a446-ba8c6847242c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:25.098123 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7c0f2-7595-42b4-a446-ba8c6847242c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:25.099135 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7c0f2-7595-42b4-a446-ba8c6847242c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:25.099756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7c0f2-7595-42b4-a446-ba8c6847242c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:25.286687 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7c0f2-7595-42b4-a446-ba8c6847242c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:25.288231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7c0f2-7595-42b4-a446-ba8c6847242c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:25.304187 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92d260-19cc-402b-8613-6cd9882a9424 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:25.345539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef11104-43be-45aa-8148-f31d3eb187fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:25.346589 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef11104-43be-45aa-8148-f31d3eb187fe None admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:25.347456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef11104-43be-45aa-8148-f31d3eb187fe None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:25.347710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef11104-43be-45aa-8148-f31d3eb187fe None admin] PATH_INFO: `/v3/projects/9799d178c2a042baa9963766f953c6a6/users/74e87dd6b03d4b6ca41070df6e3f2f11/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:25.349945 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef11104-43be-45aa-8148-f31d3eb187fe None admin] RBAC: Authorizing `identity:create_grant(project_id=9799d178c2a042baa9963766f953c6a6, user_id=74e87dd6b03d4b6ca41070df6e3f2f11, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:25.351491 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef11104-43be-45aa-8148-f31d3eb187fe None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:25.369896 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 58/115] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:40:25 2022] PUT /identity/v3/projects/9799d178c2a042baa9963766f953c6a6/users/74e87dd6b03d4b6ca41070df6e3f2f11/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:25.371050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef11104-43be-45aa-8148-f31d3eb187fe None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:25.451542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:25.462248 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 58/116] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:40:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:40:25.463692 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcdbbd-8a66-48bd-a29d-b92a707346d5 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:25.463993 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcdbbd-8a66-48bd-a29d-b92a707346d5 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:25.464276 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcdbbd-8a66-48bd-a29d-b92a707346d5 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:25.470440 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:25.472037 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 59/117] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:25.473167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bcdbbd-8a66-48bd-a29d-b92a707346d5 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:25.514259 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef11104-43be-45aa-8148-f31d3eb187fe None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:25.555097 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c61284-10f9-4418-821a-574689c80aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:25.555996 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c61284-10f9-4418-821a-574689c80aa8 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:25.556215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c61284-10f9-4418-821a-574689c80aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:25.556581 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c61284-10f9-4418-821a-574689c80aa8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:25.570823 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c61284-10f9-4418-821a-574689c80aa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:25.578991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-12c61284-10f9-4418-821a-574689c80aa8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:40:25.578991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:25.578991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:25.578991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:25.578991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:25.578991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:25.578991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:25.578991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:25.578991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:25.578991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:25.578991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:25.578991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:25.578991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:25.578991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:25.578991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:25.578991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:25.579970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:25.579970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:25.579970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:25.579970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:25.579970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:25.579970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:25.579970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:25.579970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:25.579970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:25.579970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:25.579970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:25.579970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:40:25.579970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:25.579970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:25.579970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:25.579970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:25.580962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:25.580962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:25.580962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:25.580962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:25.580962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:25.580962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:25.580962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:25.580962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:25.580962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:25.580962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:25.580962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:40:25.580962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:40:25.581580 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 59/118] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:40:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:25.582578 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bf185-44f7-4bf2-a452-3ca9b067fea0 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:25.582870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bf185-44f7-4bf2-a452-3ca9b067fea0 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:25.583156 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bf185-44f7-4bf2-a452-3ca9b067fea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:25.603646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-547bf185-44f7-4bf2-a452-3ca9b067fea0 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:25.604922 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-547bf185-44f7-4bf2-a452-3ca9b067fea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:25.697402 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 60/119] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:25.702463 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547bf185-44f7-4bf2-a452-3ca9b067fea0 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:25.714984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bcdbbd-8a66-48bd-a29d-b92a707346d5 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:25.806867 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0173f-4f4a-414b-8d26-2f2ce45d8521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:25.807566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:25.807987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0173f-4f4a-414b-8d26-2f2ce45d8521 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:25.808350 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0173f-4f4a-414b-8d26-2f2ce45d8521 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:25.808703 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0173f-4f4a-414b-8d26-2f2ce45d8521 None admin] PATH_INFO: `/v3/projects/9799d178c2a042baa9963766f953c6a6` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:25.814789 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc0173f-4f4a-414b-8d26-2f2ce45d8521 None admin] RBAC: Authorizing `identity:get_project(project_id=9799d178c2a042baa9963766f953c6a6)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:25.816472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc0173f-4f4a-414b-8d26-2f2ce45d8521 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:25.818707 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 60/120] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:25 2022] GET /identity/v3/projects/9799d178c2a042baa9963766f953c6a6 => generated 298 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:25.819544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0173f-4f4a-414b-8d26-2f2ce45d8521 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:25.858805 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547bf185-44f7-4bf2-a452-3ca9b067fea0 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:25.923062 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a886f9dc-27af-4f1d-bda2-72315bc37bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:25.924057 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886f9dc-27af-4f1d-bda2-72315bc37bc7 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:25.924475 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886f9dc-27af-4f1d-bda2-72315bc37bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:25.925450 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886f9dc-27af-4f1d-bda2-72315bc37bc7 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:25.926730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a886f9dc-27af-4f1d-bda2-72315bc37bc7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:25.927183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a886f9dc-27af-4f1d-bda2-72315bc37bc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:25.929828 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a886f9dc-27af-4f1d-bda2-72315bc37bc7 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:25.944092 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 61/121] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:25.945378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a886f9dc-27af-4f1d-bda2-72315bc37bc7 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:25.982872 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:26.065145 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0173f-4f4a-414b-8d26-2f2ce45d8521 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:26.120884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a886f9dc-27af-4f1d-bda2-72315bc37bc7 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:26.135581 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71625441-eacf-454a-bfec-b0ac59e95cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:26.137253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71625441-eacf-454a-bfec-b0ac59e95cc4 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:26.137902 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71625441-eacf-454a-bfec-b0ac59e95cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:26.138576 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71625441-eacf-454a-bfec-b0ac59e95cc4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:26.151174 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:26.162188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71625441-eacf-454a-bfec-b0ac59e95cc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:26.166450 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04adf164-93f5-4414-9ad6-20e7ca013d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:26.167227 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04adf164-93f5-4414-9ad6-20e7ca013d99 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:26.167636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04adf164-93f5-4414-9ad6-20e7ca013d99 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:26.168167 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04adf164-93f5-4414-9ad6-20e7ca013d99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:26.169179 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04adf164-93f5-4414-9ad6-20e7ca013d99 None admin] RBAC: Adding query filter params (role.id=ebb0d7a0f606450a8dbfee510b7b0b80, scope.project.id=9799d178c2a042baa9963766f953c6a6, user.id=b3413b1ed0e54a73834c597e8e23a440) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:26.169578 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04adf164-93f5-4414-9ad6-20e7ca013d99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:26.171237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04adf164-93f5-4414-9ad6-20e7ca013d99 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:26.178536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 62/122] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:40:25 2022] GET /identity/v3/role_assignments?user.id=b3413b1ed0e54a73834c597e8e23a440&scope.project.id=9799d178c2a042baa9963766f953c6a6&role.id=ebb0d7a0f606450a8dbfee510b7b0b80 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:26.179606 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04adf164-93f5-4414-9ad6-20e7ca013d99 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:26.180862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-71625441-eacf-454a-bfec-b0ac59e95cc4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:40:26.180862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:26.180862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:26.180862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:26.180862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:26.180862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:26.180862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:26.180862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:26.180862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:26.180862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:26.180862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:26.180862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:26.180862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:26.180862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:26.180862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:26.180862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:26.182755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:26.182755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:26.182755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:26.182755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:26.182755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:26.182755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:26.182755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:26.182755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:26.182755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:26.182755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:26.182755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:26.182755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:40:26.182755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:26.182755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:26.182755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:26.182755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:26.184720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:26.184720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:26.184720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:26.184720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:26.184720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:26.184720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:26.184720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:26.184720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:26.184720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:26.184720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:26.184720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:40:26.184720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:40:26.188537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 61/123] 10.222.0.78 () {56 vars in 1114 bytes} [Wed May 25 14:40:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:26.196931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd7cab-842e-4a43-bc9d-28dddbce63cf None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:26.197550 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd7cab-842e-4a43-bc9d-28dddbce63cf None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:26.198194 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd7cab-842e-4a43-bc9d-28dddbce63cf None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:26.201045 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 62/124] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:26 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:26.202756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd7cab-842e-4a43-bc9d-28dddbce63cf None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:26.336538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04adf164-93f5-4414-9ad6-20e7ca013d99 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:26.407416 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b596c928-05a6-4022-b940-a708373f0870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:26.408290 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596c928-05a6-4022-b940-a708373f0870 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:26.408573 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596c928-05a6-4022-b940-a708373f0870 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:26.408843 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596c928-05a6-4022-b940-a708373f0870 None admin] PATH_INFO: `/v3/users/f27fbbbc33b24630948a570ccae5fad0` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:26.412103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b596c928-05a6-4022-b940-a708373f0870 None admin] RBAC: Authorizing `identity:get_user(user_id=f27fbbbc33b24630948a570ccae5fad0)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:26.413779 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b596c928-05a6-4022-b940-a708373f0870 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:26.416521 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 63/125] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:26 2022] GET /identity/v3/users/f27fbbbc33b24630948a570ccae5fad0 => generated 247 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:26.419618 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855689ee-44cd-4a1c-9b1b-d9ae812a2e5d None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:26.419897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855689ee-44cd-4a1c-9b1b-d9ae812a2e5d None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:26.420174 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855689ee-44cd-4a1c-9b1b-d9ae812a2e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:26.440075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd7cab-842e-4a43-bc9d-28dddbce63cf None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:26.462512 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-855689ee-44cd-4a1c-9b1b-d9ae812a2e5d None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:26.464868 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-855689ee-44cd-4a1c-9b1b-d9ae812a2e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:26.492818 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:26.513561 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039617f-aabb-4bda-8fa3-d6da95ff8322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:26.514786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039617f-aabb-4bda-8fa3-d6da95ff8322 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:26.515467 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039617f-aabb-4bda-8fa3-d6da95ff8322 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:26.516122 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039617f-aabb-4bda-8fa3-d6da95ff8322 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:26.517085 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4039617f-aabb-4bda-8fa3-d6da95ff8322 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:26.517845 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4039617f-aabb-4bda-8fa3-d6da95ff8322 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:26.521745 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4039617f-aabb-4bda-8fa3-d6da95ff8322 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:26.539636 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 63/126] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:40:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:26.541817 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039617f-aabb-4bda-8fa3-d6da95ff8322 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:26.555825 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:26.578962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 64/127] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:26.583265 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855689ee-44cd-4a1c-9b1b-d9ae812a2e5d None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:26.827654 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855689ee-44cd-4a1c-9b1b-d9ae812a2e5d None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:26.865887 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039617f-aabb-4bda-8fa3-d6da95ff8322 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:26.945717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebff0da-5617-4bea-a3ae-33228b81d286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:26.947774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebff0da-5617-4bea-a3ae-33228b81d286 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:26.948981 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebff0da-5617-4bea-a3ae-33228b81d286 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:26.951449 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebff0da-5617-4bea-a3ae-33228b81d286 None admin] PATH_INFO: `/v3/users/74e87dd6b03d4b6ca41070df6e3f2f11` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:26.957101 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebff0da-5617-4bea-a3ae-33228b81d286 None admin] RBAC: Authorizing `identity:get_user(user_id=74e87dd6b03d4b6ca41070df6e3f2f11)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:26.961353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebff0da-5617-4bea-a3ae-33228b81d286 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:26.965443 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 65/128] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:26 2022] GET /identity/v3/users/74e87dd6b03d4b6ca41070df6e3f2f11 => generated 275 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:26.971144 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebff0da-5617-4bea-a3ae-33228b81d286 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:26.990513 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100deb7-f0aa-43c1-97ca-2e72a833723c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:26.993501 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100deb7-f0aa-43c1-97ca-2e72a833723c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:26.994614 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100deb7-f0aa-43c1-97ca-2e72a833723c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:26.995446 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100deb7-f0aa-43c1-97ca-2e72a833723c None admin] PATH_INFO: `/v3/projects/9799d178c2a042baa9963766f953c6a6` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:26.998366 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c100deb7-f0aa-43c1-97ca-2e72a833723c None admin] RBAC: Authorizing `identity:get_project(project_id=9799d178c2a042baa9963766f953c6a6)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:27.001934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c100deb7-f0aa-43c1-97ca-2e72a833723c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:27.006280 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 64/129] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:26 2022] GET /identity/v3/projects/9799d178c2a042baa9963766f953c6a6 => generated 298 bytes in 465 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:27.014408 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100deb7-f0aa-43c1-97ca-2e72a833723c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:27.099700 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:27.134574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:27.192452 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebff0da-5617-4bea-a3ae-33228b81d286 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:27.193454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100deb7-f0aa-43c1-97ca-2e72a833723c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:27.234143 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd4ac1-c494-44c9-a3d2-5c9e64e7e41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:27.235057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd4ac1-c494-44c9-a3d2-5c9e64e7e41c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:27.235349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd4ac1-c494-44c9-a3d2-5c9e64e7e41c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:27.235634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd4ac1-c494-44c9-a3d2-5c9e64e7e41c None admin] PATH_INFO: `/v3/projects/9799d178c2a042baa9963766f953c6a6` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:27.236942 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd4ac1-c494-44c9-a3d2-5c9e64e7e41c None admin] RBAC: Authorizing `identity:get_project(project_id=9799d178c2a042baa9963766f953c6a6)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:27.239055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd4ac1-c494-44c9-a3d2-5c9e64e7e41c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:27.241214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 65/130] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:27 2022] GET /identity/v3/projects/9799d178c2a042baa9963766f953c6a6 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:27.242050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd4ac1-c494-44c9-a3d2-5c9e64e7e41c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:27.271927 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda23de9-e210-4b47-a7e4-7b7e1fd5179a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:27.273941 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda23de9-e210-4b47-a7e4-7b7e1fd5179a None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:27.274730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda23de9-e210-4b47-a7e4-7b7e1fd5179a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:27.275674 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda23de9-e210-4b47-a7e4-7b7e1fd5179a None admin] PATH_INFO: `/v3/roles/member` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:27.307626 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda23de9-e210-4b47-a7e4-7b7e1fd5179a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:27.328682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-cda23de9-e210-4b47-a7e4-7b7e1fd5179a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:40:27.328682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:27.328682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:27.328682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:27.328682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:27.328682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:27.328682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:27.328682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:27.328682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:27.328682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:27.328682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:27.328682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:27.328682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:27.328682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:27.328682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:27.328682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:27.330904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:27.330904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:27.330904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:27.330904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:27.330904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:27.330904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:27.330904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:27.330904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:27.330904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:27.330904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:27.330904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:27.330904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:40:27.330904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:27.330904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:27.330904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:27.330904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:27.332974 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:27.332974 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:27.332974 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:27.332974 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:27.332974 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:27.332974 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:27.332974 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:27.332974 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:27.332974 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:27.332974 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:27.332974 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:40:27.332974 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:40:27.335216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 66/131] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:27.341481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860eaa35-bbd4-441f-9065-fbfea5cac9eb None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:27.341481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860eaa35-bbd4-441f-9065-fbfea5cac9eb None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:27.341481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860eaa35-bbd4-441f-9065-fbfea5cac9eb None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:27.341481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 67/132] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:27.341481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860eaa35-bbd4-441f-9065-fbfea5cac9eb None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:27.393147 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd4ac1-c494-44c9-a3d2-5c9e64e7e41c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:27.429888 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:27.454788 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8437dc1-13b2-4491-a4ea-f193a205a38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:27.456535 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8437dc1-13b2-4491-a4ea-f193a205a38e None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:27.457054 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8437dc1-13b2-4491-a4ea-f193a205a38e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:27.457568 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8437dc1-13b2-4491-a4ea-f193a205a38e None admin] PATH_INFO: `/v3/projects/9799d178c2a042baa9963766f953c6a6/users/f27fbbbc33b24630948a570ccae5fad0/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:27.461078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8437dc1-13b2-4491-a4ea-f193a205a38e None admin] RBAC: Authorizing `identity:create_grant(project_id=9799d178c2a042baa9963766f953c6a6, user_id=f27fbbbc33b24630948a570ccae5fad0, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:27.463070 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8437dc1-13b2-4491-a4ea-f193a205a38e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:27.491303 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 66/133] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:40:27 2022] PUT /identity/v3/projects/9799d178c2a042baa9963766f953c6a6/users/f27fbbbc33b24630948a570ccae5fad0/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:27.493491 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f1ca4-389f-4624-a592-a2bfb90be7de None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:27.494676 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f1ca4-389f-4624-a592-a2bfb90be7de None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:27.494676 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f1ca4-389f-4624-a592-a2bfb90be7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:27.496939 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860eaa35-bbd4-441f-9065-fbfea5cac9eb None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:27.520611 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-136f1ca4-389f-4624-a592-a2bfb90be7de None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:27.522607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-136f1ca4-389f-4624-a592-a2bfb90be7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:27.528343 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:27.619576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fed012-655c-43a5-a662-8ce33bc07198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:27.620751 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fed012-655c-43a5-a662-8ce33bc07198 None admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:27.620895 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fed012-655c-43a5-a662-8ce33bc07198 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:27.621082 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fed012-655c-43a5-a662-8ce33bc07198 None admin] PATH_INFO: `/v3/projects/9799d178c2a042baa9963766f953c6a6/users/74e87dd6b03d4b6ca41070df6e3f2f11/roles/d451467ccbbf4149abbfc9b0f76b9339` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:27.630706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 67/134] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:27.631035 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fed012-655c-43a5-a662-8ce33bc07198 None admin] RBAC: Authorizing `identity:create_grant(project_id=9799d178c2a042baa9963766f953c6a6, user_id=74e87dd6b03d4b6ca41070df6e3f2f11, role_id=d451467ccbbf4149abbfc9b0f76b9339)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:27.632320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136f1ca4-389f-4624-a592-a2bfb90be7de None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:27.632536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fed012-655c-43a5-a662-8ce33bc07198 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:27.680805 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 68/135] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:40:27 2022] PUT /identity/v3/projects/9799d178c2a042baa9963766f953c6a6/users/74e87dd6b03d4b6ca41070df6e3f2f11/roles/d451467ccbbf4149abbfc9b0f76b9339 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:27.726285 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fed012-655c-43a5-a662-8ce33bc07198 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:27.779159 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136f1ca4-389f-4624-a592-a2bfb90be7de None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:27.799656 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:27.837224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2039484-6310-4070-ad24-4eb13561f485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:27.838593 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2039484-6310-4070-ad24-4eb13561f485 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:27.839335 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2039484-6310-4070-ad24-4eb13561f485 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:27.839853 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2039484-6310-4070-ad24-4eb13561f485 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:27.840903 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2039484-6310-4070-ad24-4eb13561f485 None admin] RBAC: Adding query filter params (name=member) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:27.841703 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2039484-6310-4070-ad24-4eb13561f485 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:27.843832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2039484-6310-4070-ad24-4eb13561f485 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:27.856129 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 68/136] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:40:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:27.869704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2039484-6310-4070-ad24-4eb13561f485 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:28.001377 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fed012-655c-43a5-a662-8ce33bc07198 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:28.048003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2039484-6310-4070-ad24-4eb13561f485 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:28.058827 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff1ff2-180c-43d7-8fa1-1e23c071ab13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:28.059949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff1ff2-180c-43d7-8fa1-1e23c071ab13 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:28.059949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff1ff2-180c-43d7-8fa1-1e23c071ab13 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:28.060246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff1ff2-180c-43d7-8fa1-1e23c071ab13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:28.061662 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff1ff2-180c-43d7-8fa1-1e23c071ab13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:28.063424 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff1ff2-180c-43d7-8fa1-1e23c071ab13 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:28.065278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 69/137] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:40:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:28.071876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff1ff2-180c-43d7-8fa1-1e23c071ab13 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:28.097158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095208d-deeb-46d5-b38d-543b2dbabe7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:28.098449 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095208d-deeb-46d5-b38d-543b2dbabe7c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:28.098943 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095208d-deeb-46d5-b38d-543b2dbabe7c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:28.099501 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095208d-deeb-46d5-b38d-543b2dbabe7c None admin] PATH_INFO: `/v3/users/74e87dd6b03d4b6ca41070df6e3f2f11` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:28.101233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095208d-deeb-46d5-b38d-543b2dbabe7c None admin] RBAC: Authorizing `identity:get_user(user_id=74e87dd6b03d4b6ca41070df6e3f2f11)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:28.103233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095208d-deeb-46d5-b38d-543b2dbabe7c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:28.105492 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 69/138] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:27 2022] GET /identity/v3/users/74e87dd6b03d4b6ca41070df6e3f2f11 => generated 275 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:28.117981 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095208d-deeb-46d5-b38d-543b2dbabe7c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:28.142548 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:28.196640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:28.275297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff1ff2-180c-43d7-8fa1-1e23c071ab13 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:28.306464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095208d-deeb-46d5-b38d-543b2dbabe7c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:28.323187 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d314c46-f59d-4229-9c44-2dda782663ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:28.323892 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d314c46-f59d-4229-9c44-2dda782663ac None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:28.324176 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d314c46-f59d-4229-9c44-2dda782663ac None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:28.324458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d314c46-f59d-4229-9c44-2dda782663ac None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:28.325205 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d314c46-f59d-4229-9c44-2dda782663ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:28.327720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d314c46-f59d-4229-9c44-2dda782663ac None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:28.357568 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 70/139] 10.222.0.78 () {60 vars in 1132 bytes} [Wed May 25 14:40:28 2022] POST /identity/v3/users => generated 283 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:40:28.391082 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5b543-efdd-433d-b19d-68787d292735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:28.392847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5b543-efdd-433d-b19d-68787d292735 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:28.393893 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5b543-efdd-433d-b19d-68787d292735 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:28.394488 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5b543-efdd-433d-b19d-68787d292735 None admin] PATH_INFO: `/v3/projects/39ae31a66cc64d0095d6f865e887f407` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:28.396798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5b543-efdd-433d-b19d-68787d292735 None admin] RBAC: Authorizing `identity:get_project(project_id=39ae31a66cc64d0095d6f865e887f407)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:28.400044 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5b543-efdd-433d-b19d-68787d292735 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:28.404009 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 70/140] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:28 2022] GET /identity/v3/projects/39ae31a66cc64d0095d6f865e887f407 => generated 312 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:28.411174 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d314c46-f59d-4229-9c44-2dda782663ac None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:28.573625 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:28.576007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d314c46-f59d-4229-9c44-2dda782663ac None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:28.618591 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ce28f-2050-40fc-a764-4ee94a41392e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:28.619532 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ce28f-2050-40fc-a764-4ee94a41392e None admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:28.619815 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ce28f-2050-40fc-a764-4ee94a41392e None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:28.620100 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ce28f-2050-40fc-a764-4ee94a41392e None admin] PATH_INFO: `/v3/projects/39ae31a66cc64d0095d6f865e887f407/users/74e87dd6b03d4b6ca41070df6e3f2f11/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:28.622448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ce28f-2050-40fc-a764-4ee94a41392e None admin] RBAC: Authorizing `identity:create_grant(project_id=39ae31a66cc64d0095d6f865e887f407, user_id=74e87dd6b03d4b6ca41070df6e3f2f11, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:28.624281 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ce28f-2050-40fc-a764-4ee94a41392e None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:28.646480 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 71/141] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:40:28 2022] PUT /identity/v3/projects/39ae31a66cc64d0095d6f865e887f407/users/74e87dd6b03d4b6ca41070df6e3f2f11/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:40:28.760539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cdacd1-2877-480c-9172-ebfbbfa6c178 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:28.760885 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cdacd1-2877-480c-9172-ebfbbfa6c178 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:28.761110 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cdacd1-2877-480c-9172-ebfbbfa6c178 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:28.762636 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 71/142] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:28.769415 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef60f2-1109-48d6-8ab4-f39830053d5f None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:28.769866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef60f2-1109-48d6-8ab4-f39830053d5f None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:28.770297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef60f2-1109-48d6-8ab4-f39830053d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:28.789872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-99ef60f2-1109-48d6-8ab4-f39830053d5f None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:28.791178 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-99ef60f2-1109-48d6-8ab4-f39830053d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:28.877264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 72/143] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:28.903551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cdacd1-2877-480c-9172-ebfbbfa6c178 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:29.057365 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cdacd1-2877-480c-9172-ebfbbfa6c178 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:29.070830 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:29.106449 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53f5d5-969a-4115-8a39-101735faee19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:29.107276 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53f5d5-969a-4115-8a39-101735faee19 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:29.107563 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53f5d5-969a-4115-8a39-101735faee19 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:29.107916 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53f5d5-969a-4115-8a39-101735faee19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:29.120880 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c53f5d5-969a-4115-8a39-101735faee19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:29.130126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-8c53f5d5-969a-4115-8a39-101735faee19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:40:29.130126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:29.130126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:29.130126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:29.130126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:29.130126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:29.130126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:29.130126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:29.130126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:29.130126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:29.130126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:29.130126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:29.130126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:29.130126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:29.130126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:29.130126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:29.131086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:29.131086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:29.131086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:29.131086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:29.131086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:29.131086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:29.131086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:29.131086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:29.131086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:29.131086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:29.131086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:29.131086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:40:29.131086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:29.131086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:29.131086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:29.131086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:29.131824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:29.131824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:29.131824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:29.131824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:29.131824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:29.131824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:29.131824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:29.131824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:29.131824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:29.131824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:29.131824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:40:29.131824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:40:29.131824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 72/144] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:40:29.137111 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef60f2-1109-48d6-8ab4-f39830053d5f None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:29.250671 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:29.264479 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef60f2-1109-48d6-8ab4-f39830053d5f None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:29.314902 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda45bc-1d83-4109-bc0a-51f55c43ab2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:29.316297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda45bc-1d83-4109-bc0a-51f55c43ab2c None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:29.316650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda45bc-1d83-4109-bc0a-51f55c43ab2c None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:29.316998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda45bc-1d83-4109-bc0a-51f55c43ab2c None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:29.317971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda45bc-1d83-4109-bc0a-51f55c43ab2c None admin] RBAC: Adding query filter params (name=member) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:29.318377 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda45bc-1d83-4109-bc0a-51f55c43ab2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:29.319947 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda45bc-1d83-4109-bc0a-51f55c43ab2c None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:29.331611 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 73/145] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:40:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:29.345503 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53f5d5-969a-4115-8a39-101735faee19 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:29.535063 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53f5d5-969a-4115-8a39-101735faee19 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:29.597236 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865c2e8-c85c-4d12-ac84-73462261c45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:29.598220 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865c2e8-c85c-4d12-ac84-73462261c45d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:29.598471 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865c2e8-c85c-4d12-ac84-73462261c45d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:29.598755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865c2e8-c85c-4d12-ac84-73462261c45d None admin] PATH_INFO: `/v3/users/74e87dd6b03d4b6ca41070df6e3f2f11` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:29.604491 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865c2e8-c85c-4d12-ac84-73462261c45d None admin] RBAC: Authorizing `identity:get_user(user_id=74e87dd6b03d4b6ca41070df6e3f2f11)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:29.606037 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865c2e8-c85c-4d12-ac84-73462261c45d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:29.608557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 73/146] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:29 2022] GET /identity/v3/users/74e87dd6b03d4b6ca41070df6e3f2f11 => generated 275 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:29.618309 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda45bc-1d83-4109-bc0a-51f55c43ab2c None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:29.782916 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda45bc-1d83-4109-bc0a-51f55c43ab2c None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:29.876425 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1958623-f0d4-465f-b9e4-b3d7473d55e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:29.877006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1958623-f0d4-465f-b9e4-b3d7473d55e9 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:29.877287 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1958623-f0d4-465f-b9e4-b3d7473d55e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:29.877560 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1958623-f0d4-465f-b9e4-b3d7473d55e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:29.915924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:30.020578 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1958623-f0d4-465f-b9e4-b3d7473d55e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:30.022474 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1958623-f0d4-465f-b9e4-b3d7473d55e9 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:30.176358 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 74/147] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:40:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:40:30.181673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865c2e8-c85c-4d12-ac84-73462261c45d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:30.268268 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:30.352884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865c2e8-c85c-4d12-ac84-73462261c45d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:30.417402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4841e7-4087-4702-8427-afaf5e6f621d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:30.418312 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4841e7-4087-4702-8427-afaf5e6f621d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:30.418602 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4841e7-4087-4702-8427-afaf5e6f621d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:30.418887 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4841e7-4087-4702-8427-afaf5e6f621d None admin] PATH_INFO: `/v3/projects/9799d178c2a042baa9963766f953c6a6` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:30.420108 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4841e7-4087-4702-8427-afaf5e6f621d None admin] RBAC: Authorizing `identity:get_project(project_id=9799d178c2a042baa9963766f953c6a6)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:30.421648 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4841e7-4087-4702-8427-afaf5e6f621d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:30.423479 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 74/148] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:30 2022] GET /identity/v3/projects/9799d178c2a042baa9963766f953c6a6 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:30.430064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1958623-f0d4-465f-b9e4-b3d7473d55e9 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:30.488145 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3598250-21a5-49b7-a87a-d96dc0eb9706 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:30.488478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3598250-21a5-49b7-a87a-d96dc0eb9706 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:30.488855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3598250-21a5-49b7-a87a-d96dc0eb9706 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:30.490612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 75/149] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:30.495501 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:30.497173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be0d97-f691-4252-b85c-9af989a2acd3 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:30.497525 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be0d97-f691-4252-b85c-9af989a2acd3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:30.497810 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be0d97-f691-4252-b85c-9af989a2acd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:30.520912 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-c4be0d97-f691-4252-b85c-9af989a2acd3 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:30.522160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c4be0d97-f691-4252-b85c-9af989a2acd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:30.648167 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1958623-f0d4-465f-b9e4-b3d7473d55e9 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:30.715524 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 76/150] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:30.763721 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd9942-b9f9-4989-99db-feb8c582635e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:30.764837 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd9942-b9f9-4989-99db-feb8c582635e None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:30.765435 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd9942-b9f9-4989-99db-feb8c582635e None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:30.765911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd9942-b9f9-4989-99db-feb8c582635e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:30.766493 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be0d97-f691-4252-b85c-9af989a2acd3 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:30.767488 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd9942-b9f9-4989-99db-feb8c582635e None admin] RBAC: Adding query filter params (role.id=a3da3c62738d4790b9dac512b1568b52, scope.project.id=9799d178c2a042baa9963766f953c6a6, user.id=74e87dd6b03d4b6ca41070df6e3f2f11) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:30.768055 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd9942-b9f9-4989-99db-feb8c582635e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:30.772492 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd9942-b9f9-4989-99db-feb8c582635e None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:30.792571 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 75/151] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:40:30 2022] GET /identity/v3/role_assignments?user.id=74e87dd6b03d4b6ca41070df6e3f2f11&scope.project.id=9799d178c2a042baa9963766f953c6a6&role.id=a3da3c62738d4790b9dac512b1568b52 => generated 608 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:30.844574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:30.965850 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be0d97-f691-4252-b85c-9af989a2acd3 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:31.028938 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26366ba3-c15d-4177-b114-ad49dd019fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:31.029749 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26366ba3-c15d-4177-b114-ad49dd019fa3 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:31.030052 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26366ba3-c15d-4177-b114-ad49dd019fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:31.030340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26366ba3-c15d-4177-b114-ad49dd019fa3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:31.045314 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26366ba3-c15d-4177-b114-ad49dd019fa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:31.059609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-26366ba3-c15d-4177-b114-ad49dd019fa3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:40:31.059609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:31.059609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:31.059609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:31.059609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:31.059609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:31.059609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:31.059609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:31.059609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:31.059609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:31.059609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:31.059609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:31.059609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:31.059609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:31.059609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:31.059609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:31.060528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:31.060528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:31.060528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:31.060528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:31.060528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:31.060528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:31.060528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:31.060528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:31.060528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:31.060528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:31.060528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:31.060528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:40:31.060528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:31.060528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:31.060528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:31.060528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:31.061297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:31.061297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:31.061297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:31.061297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:31.061297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:31.061297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:31.061297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:31.061297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:31.061297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:31.061297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:31.061297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:40:31.061297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:40:31.061297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 77/152] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:31.071827 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd9942-b9f9-4989-99db-feb8c582635e None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:31.287386 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd9942-b9f9-4989-99db-feb8c582635e None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:31.303979 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:31.331031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a29e4-c57e-42b5-a9e0-953c13ec4b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:31.331732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a29e4-c57e-42b5-a9e0-953c13ec4b8f None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:31.332008 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a29e4-c57e-42b5-a9e0-953c13ec4b8f None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:31.332282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a29e4-c57e-42b5-a9e0-953c13ec4b8f None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:31.333066 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a29e4-c57e-42b5-a9e0-953c13ec4b8f None admin] RBAC: Adding query filter params (name=reader) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:31.333376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a29e4-c57e-42b5-a9e0-953c13ec4b8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:31.339078 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a29e4-c57e-42b5-a9e0-953c13ec4b8f None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:31.348621 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 76/153] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:40:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:31.358470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26366ba3-c15d-4177-b114-ad49dd019fa3 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:31.531876 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:31.538808 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26366ba3-c15d-4177-b114-ad49dd019fa3 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:31.591226 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064b29f-440c-43ce-9ac7-c6456d4eb7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:31.592126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064b29f-440c-43ce-9ac7-c6456d4eb7d7 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:31.592414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064b29f-440c-43ce-9ac7-c6456d4eb7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:31.592690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064b29f-440c-43ce-9ac7-c6456d4eb7d7 None admin] PATH_INFO: `/v3/users/b3413b1ed0e54a73834c597e8e23a440` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:31.599017 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e064b29f-440c-43ce-9ac7-c6456d4eb7d7 None admin] RBAC: Authorizing `identity:get_user(user_id=b3413b1ed0e54a73834c597e8e23a440)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:31.600573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e064b29f-440c-43ce-9ac7-c6456d4eb7d7 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:31.602413 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 78/154] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:31 2022] GET /identity/v3/users/b3413b1ed0e54a73834c597e8e23a440 => generated 289 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:31.608627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a29e4-c57e-42b5-a9e0-953c13ec4b8f None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:31.844804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a29e4-c57e-42b5-a9e0-953c13ec4b8f None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:31.914902 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d473ef09-e26c-44d5-aac2-5d8aa15565cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:31.916401 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473ef09-e26c-44d5-aac2-5d8aa15565cb None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:31.916892 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473ef09-e26c-44d5-aac2-5d8aa15565cb None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:31.917391 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473ef09-e26c-44d5-aac2-5d8aa15565cb None admin] PATH_INFO: `/v3/projects/9799d178c2a042baa9963766f953c6a6` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:31.919722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d473ef09-e26c-44d5-aac2-5d8aa15565cb None admin] RBAC: Authorizing `identity:get_project(project_id=9799d178c2a042baa9963766f953c6a6)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:31.922573 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d473ef09-e26c-44d5-aac2-5d8aa15565cb None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:31.926992 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 77/155] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:31 2022] GET /identity/v3/projects/9799d178c2a042baa9963766f953c6a6 => generated 298 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:31.934837 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064b29f-440c-43ce-9ac7-c6456d4eb7d7 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:32.163389 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064b29f-440c-43ce-9ac7-c6456d4eb7d7 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:32.242741 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cbe25f-638c-4211-a28f-6904be1e883b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:32.244364 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbe25f-638c-4211-a28f-6904be1e883b None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:32.244882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbe25f-638c-4211-a28f-6904be1e883b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:32.245402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbe25f-638c-4211-a28f-6904be1e883b None admin] PATH_INFO: `/v3/projects/9799d178c2a042baa9963766f953c6a6/users/b3413b1ed0e54a73834c597e8e23a440/roles/ebb0d7a0f606450a8dbfee510b7b0b80` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:32.252684 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cbe25f-638c-4211-a28f-6904be1e883b None admin] RBAC: Authorizing `identity:create_grant(project_id=9799d178c2a042baa9963766f953c6a6, user_id=b3413b1ed0e54a73834c597e8e23a440, role_id=ebb0d7a0f606450a8dbfee510b7b0b80)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:32.256498 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cbe25f-638c-4211-a28f-6904be1e883b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:32.297348 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 79/156] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:40:31 2022] PUT /identity/v3/projects/9799d178c2a042baa9963766f953c6a6/users/b3413b1ed0e54a73834c597e8e23a440/roles/ebb0d7a0f606450a8dbfee510b7b0b80 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:32.431556 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8a935-2bd6-4f02-9ae8-87e631feeeec None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:32.432155 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8a935-2bd6-4f02-9ae8-87e631feeeec None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:32.432641 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8a935-2bd6-4f02-9ae8-87e631feeeec None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:32.434300 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 78/157] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:32.441990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af27fd-f4b8-4f8f-a150-8e172bf40155 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:32.442548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af27fd-f4b8-4f8f-a150-8e172bf40155 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:32.442989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af27fd-f4b8-4f8f-a150-8e172bf40155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:32.466350 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b5af27fd-f4b8-4f8f-a150-8e172bf40155 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:32.467559 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b5af27fd-f4b8-4f8f-a150-8e172bf40155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:32.500624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:32.574889 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 80/158] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:32.611599 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d8a935-2bd6-4f02-9ae8-87e631feeeec None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:32.693426 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:32.775063 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d8a935-2bd6-4f02-9ae8-87e631feeeec None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:32.829397 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc6d71-ddd9-4229-97c9-670a63e42d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:32.831389 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc6d71-ddd9-4229-97c9-670a63e42d99 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:32.831954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc6d71-ddd9-4229-97c9-670a63e42d99 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:32.832590 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc6d71-ddd9-4229-97c9-670a63e42d99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:32.849824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc6d71-ddd9-4229-97c9-670a63e42d99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:32.858264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-25cc6d71-ddd9-4229-97c9-670a63e42d99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:40:32.858264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:32.858264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:32.858264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:32.858264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:32.858264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:32.858264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:32.858264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:32.858264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:32.858264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:32.858264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:32.858264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:32.858264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:32.858264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:32.858264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:32.858264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:32.859229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:32.859229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:32.859229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:32.859229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:32.859229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:32.859229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:32.859229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:32.859229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:32.859229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:32.859229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:32.859229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:32.859229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:40:32.859229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:32.859229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:32.859229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:32.859229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:32.859979 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:32.859979 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:32.859979 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:32.859979 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:32.859979 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:32.859979 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:32.859979 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:32.859979 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:32.859979 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:32.859979 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:32.859979 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:40:32.859979 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:40:32.863388 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 79/159] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:40:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:32.869897 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af27fd-f4b8-4f8f-a150-8e172bf40155 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:33.036715 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af27fd-f4b8-4f8f-a150-8e172bf40155 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:33.083517 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273af41-7e6e-4707-873b-38c71d7aebb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:33.084207 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273af41-7e6e-4707-873b-38c71d7aebb3 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:33.084494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273af41-7e6e-4707-873b-38c71d7aebb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:33.084788 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273af41-7e6e-4707-873b-38c71d7aebb3 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:33.085561 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273af41-7e6e-4707-873b-38c71d7aebb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:33.085866 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273af41-7e6e-4707-873b-38c71d7aebb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:33.087332 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273af41-7e6e-4707-873b-38c71d7aebb3 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:33.100409 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 81/160] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:33.106749 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc6d71-ddd9-4229-97c9-670a63e42d99 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:33.186743 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc215d-db9d-46cd-8f55-0399af3250a0 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:33.187386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc215d-db9d-46cd-8f55-0399af3250a0 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:33.187862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc215d-db9d-46cd-8f55-0399af3250a0 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:33.189535 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 82/161] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:40:33.195342 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225cc70-8add-4d34-b42e-b99c1dc59935 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:33.195975 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225cc70-8add-4d34-b42e-b99c1dc59935 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:33.196444 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225cc70-8add-4d34-b42e-b99c1dc59935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:33.239239 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-5225cc70-8add-4d34-b42e-b99c1dc59935 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:33.240849 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5225cc70-8add-4d34-b42e-b99c1dc59935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:33.270442 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:33.328389 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc6d71-ddd9-4229-97c9-670a63e42d99 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:33.335474 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:33.407825 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb678f6c-9d09-4a3d-8d35-e5a879a4e447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:33.408718 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb678f6c-9d09-4a3d-8d35-e5a879a4e447 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:33.409539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb678f6c-9d09-4a3d-8d35-e5a879a4e447 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:33.409807 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb678f6c-9d09-4a3d-8d35-e5a879a4e447 None admin] PATH_INFO: `/v3/users/f27fbbbc33b24630948a570ccae5fad0` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:33.411106 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb678f6c-9d09-4a3d-8d35-e5a879a4e447 None admin] RBAC: Authorizing `identity:get_user(user_id=f27fbbbc33b24630948a570ccae5fad0)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:33.413139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb678f6c-9d09-4a3d-8d35-e5a879a4e447 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:33.415543 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 80/162] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:33 2022] GET /identity/v3/users/f27fbbbc33b24630948a570ccae5fad0 => generated 247 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:33.423104 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb678f6c-9d09-4a3d-8d35-e5a879a4e447 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:33.430519 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 83/163] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:33.472910 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225cc70-8add-4d34-b42e-b99c1dc59935 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:33.636106 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb678f6c-9d09-4a3d-8d35-e5a879a4e447 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:33.723994 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a658aa4-71e9-4c0b-a06d-a37be319b2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:33.725911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a658aa4-71e9-4c0b-a06d-a37be319b2d5 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:33.727243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a658aa4-71e9-4c0b-a06d-a37be319b2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:33.727700 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a658aa4-71e9-4c0b-a06d-a37be319b2d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:33.744003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225cc70-8add-4d34-b42e-b99c1dc59935 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:33.824822 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:33.881404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a658aa4-71e9-4c0b-a06d-a37be319b2d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:33.883010 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a658aa4-71e9-4c0b-a06d-a37be319b2d5 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:33.911289 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56937e7-c349-48a8-8958-96afd3bdd633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:33.912452 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56937e7-c349-48a8-8958-96afd3bdd633 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:33.912749 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56937e7-c349-48a8-8958-96afd3bdd633 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:33.913116 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56937e7-c349-48a8-8958-96afd3bdd633 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:33.959425 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56937e7-c349-48a8-8958-96afd3bdd633 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:33.981518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-b56937e7-c349-48a8-8958-96afd3bdd633 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:40:33.981518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:33.981518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:33.981518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:33.981518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:33.981518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:33.981518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:33.981518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:33.981518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:33.981518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:33.981518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:33.981518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:33.981518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:33.981518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:33.981518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:33.981518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:33.982652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:33.982652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:33.982652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:33.982652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:33.982652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:33.982652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:33.982652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:33.982652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:33.982652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:33.982652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:33.982652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:33.982652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:40:33.982652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:33.982652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:33.982652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:33.982652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:33.984791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:33.984791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:33.984791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:33.984791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:33.984791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:33.984791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:33.984791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:33.984791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:33.984791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:33.984791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:33.984791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:40:33.984791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:40:33.984791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 84/164] 10.222.0.78 () {56 vars in 1114 bytes} [Wed May 25 14:40:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 512 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:40:33.987743 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4d6b5-ae56-4327-8be0-3360e5b411e4 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:33.987743 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4d6b5-ae56-4327-8be0-3360e5b411e4 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:33.990556 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4d6b5-ae56-4327-8be0-3360e5b411e4 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:33.992809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 85/165] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:33.994347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4d6b5-ae56-4327-8be0-3360e5b411e4 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:34.018546 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:34.166291 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4d6b5-ae56-4327-8be0-3360e5b411e4 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:34.187480 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 81/166] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:40:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 765 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:40:34.188901 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6bf834-a485-4559-8522-abac5dc0420c None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:34.189183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6bf834-a485-4559-8522-abac5dc0420c None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:34.189460 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6bf834-a485-4559-8522-abac5dc0420c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:34.207511 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07458f-6592-4a45-9963-b5261a965c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:34.208282 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07458f-6592-4a45-9963-b5261a965c89 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:34.208576 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07458f-6592-4a45-9963-b5261a965c89 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:34.208858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07458f-6592-4a45-9963-b5261a965c89 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:34.209654 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07458f-6592-4a45-9963-b5261a965c89 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:34.209971 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07458f-6592-4a45-9963-b5261a965c89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:34.215707 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07458f-6592-4a45-9963-b5261a965c89 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:34.222485 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 86/167] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:40:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:34.223410 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07458f-6592-4a45-9963-b5261a965c89 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:34.228820 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-cd6bf834-a485-4559-8522-abac5dc0420c None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:34.230103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-cd6bf834-a485-4559-8522-abac5dc0420c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:34.313807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:34.358836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07458f-6592-4a45-9963-b5261a965c89 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:34.399254 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f9d48-0ea3-4b61-a70a-53de0611d242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:34.400427 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f9d48-0ea3-4b61-a70a-53de0611d242 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:34.400861 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f9d48-0ea3-4b61-a70a-53de0611d242 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:34.401285 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f9d48-0ea3-4b61-a70a-53de0611d242 None admin] PATH_INFO: `/v3/projects/9799d178c2a042baa9963766f953c6a6` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:34.402766 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8f9d48-0ea3-4b61-a70a-53de0611d242 None admin] RBAC: Authorizing `identity:get_project(project_id=9799d178c2a042baa9963766f953c6a6)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:34.404407 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8f9d48-0ea3-4b61-a70a-53de0611d242 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:34.406379 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 87/168] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:34 2022] GET /identity/v3/projects/9799d178c2a042baa9963766f953c6a6 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:34.407332 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f9d48-0ea3-4b61-a70a-53de0611d242 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:34.500513 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 82/169] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:34.502374 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6bf834-a485-4559-8522-abac5dc0420c None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:34.510399 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:34.523023 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f9d48-0ea3-4b61-a70a-53de0611d242 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:34.565989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004c588-2e44-48da-9f0c-1a3dd8393d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:34.567171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004c588-2e44-48da-9f0c-1a3dd8393d06 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:34.567605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004c588-2e44-48da-9f0c-1a3dd8393d06 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:34.568026 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004c588-2e44-48da-9f0c-1a3dd8393d06 None admin] PATH_INFO: `/v3/users/74e87dd6b03d4b6ca41070df6e3f2f11` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:34.569587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004c588-2e44-48da-9f0c-1a3dd8393d06 None admin] RBAC: Authorizing `identity:get_user(user_id=74e87dd6b03d4b6ca41070df6e3f2f11)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:34.571356 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004c588-2e44-48da-9f0c-1a3dd8393d06 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:34.573272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 88/170] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:34 2022] GET /identity/v3/users/74e87dd6b03d4b6ca41070df6e3f2f11 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:34.574248 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004c588-2e44-48da-9f0c-1a3dd8393d06 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:34.578516 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:34.705643 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004c588-2e44-48da-9f0c-1a3dd8393d06 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:34.747650 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f17d6-f95f-45c3-bb83-8b4834ef2fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:34.748741 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f17d6-f95f-45c3-bb83-8b4834ef2fdf None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:34.749184 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f17d6-f95f-45c3-bb83-8b4834ef2fdf None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:34.749667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f17d6-f95f-45c3-bb83-8b4834ef2fdf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:34.763768 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577f17d6-f95f-45c3-bb83-8b4834ef2fdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:34.773080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-577f17d6-f95f-45c3-bb83-8b4834ef2fdf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:40:34.773080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:34.773080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:34.773080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:34.773080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:34.773080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:34.773080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:34.773080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:34.773080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:34.773080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:34.773080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:34.773080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:34.773080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:34.773080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:34.773080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:34.773080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:34.774234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:34.774234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:34.774234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:34.774234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:34.774234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:34.774234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:34.774234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:34.774234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:34.774234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:34.774234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:34.774234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:34.774234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:40:34.774234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:34.774234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:34.774234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:34.774234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:34.775173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:34.775173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:34.775173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:34.775173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:34.775173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:34.775173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:34.775173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:34.775173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:34.775173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:34.775173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:34.775173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:40:34.775173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:40:34.776472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 89/171] 10.222.0.78 () {56 vars in 1114 bytes} [Wed May 25 14:40:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:34.781282 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f17d6-f95f-45c3-bb83-8b4834ef2fdf None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:34.798725 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6bf834-a485-4559-8522-abac5dc0420c None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:34.853078 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e20b32-8cfb-40b1-8d27-342d51f69365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:34.853758 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e20b32-8cfb-40b1-8d27-342d51f69365 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:34.854050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e20b32-8cfb-40b1-8d27-342d51f69365 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:34.854328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e20b32-8cfb-40b1-8d27-342d51f69365 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:34.855513 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e20b32-8cfb-40b1-8d27-342d51f69365 None admin] RBAC: Adding query filter params (role.id=e566b37cd92f467198b5dc40f183effc, scope.project.id=9799d178c2a042baa9963766f953c6a6, user.id=f27fbbbc33b24630948a570ccae5fad0) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:34.855820 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e20b32-8cfb-40b1-8d27-342d51f69365 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:34.858237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e20b32-8cfb-40b1-8d27-342d51f69365 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:34.860905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:34.870106 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 83/172] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:40:34 2022] GET /identity/v3/role_assignments?user.id=f27fbbbc33b24630948a570ccae5fad0&scope.project.id=9799d178c2a042baa9963766f953c6a6&role.id=e566b37cd92f467198b5dc40f183effc => generated 608 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:34.872100 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b983d83-99ea-46b7-8784-e30c38cc40f1 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:34.872654 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b983d83-99ea-46b7-8784-e30c38cc40f1 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:34.873142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b983d83-99ea-46b7-8784-e30c38cc40f1 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:34.875202 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 84/173] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:34.880162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ab2ea-3913-484b-9557-36a583274b6b None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:34.886572 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ab2ea-3913-484b-9557-36a583274b6b None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:34.887196 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ab2ea-3913-484b-9557-36a583274b6b None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:34.889041 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 85/174] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:34 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:34.892604 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ab2ea-3913-484b-9557-36a583274b6b None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:34.958925 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f17d6-f95f-45c3-bb83-8b4834ef2fdf None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:35.043876 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4f9be-ea2b-4f55-b50f-9c1af499737c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:35.045471 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4f9be-ea2b-4f55-b50f-9c1af499737c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:35.046003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4f9be-ea2b-4f55-b50f-9c1af499737c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:35.046513 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4f9be-ea2b-4f55-b50f-9c1af499737c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:35.057726 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:35.128180 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ab2ea-3913-484b-9557-36a583274b6b None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:35.173278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af66e6-7bab-40b2-9cb2-067eb422efe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:35.174550 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af66e6-7bab-40b2-9cb2-067eb422efe2 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:35.175048 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af66e6-7bab-40b2-9cb2-067eb422efe2 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:35.175562 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af66e6-7bab-40b2-9cb2-067eb422efe2 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:35.176646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af66e6-7bab-40b2-9cb2-067eb422efe2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:35.177188 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af66e6-7bab-40b2-9cb2-067eb422efe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:35.178887 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af66e6-7bab-40b2-9cb2-067eb422efe2 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:35.182712 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f4f9be-ea2b-4f55-b50f-9c1af499737c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:35.184974 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f4f9be-ea2b-4f55-b50f-9c1af499737c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:35.190244 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 86/175] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:40:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:35.191624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c0d73-0537-4f09-8754-225d80c72869 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:35.191919 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c0d73-0537-4f09-8754-225d80c72869 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:35.192199 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c0d73-0537-4f09-8754-225d80c72869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:35.220695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-b14c0d73-0537-4f09-8754-225d80c72869 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:35.224708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-b14c0d73-0537-4f09-8754-225d80c72869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:35.308661 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:35.355160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 90/176] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:40:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:40:35.356635 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c38d7c-8f7a-46d2-a8a9-2a272aa5cf01 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:35.357165 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c38d7c-8f7a-46d2-a8a9-2a272aa5cf01 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:35.357522 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c38d7c-8f7a-46d2-a8a9-2a272aa5cf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:35.382049 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 87/177] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:35.382049 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b37a0-2528-4311-ba0f-ed43bb633465 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:35.382049 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b37a0-2528-4311-ba0f-ed43bb633465 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:35.382049 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b37a0-2528-4311-ba0f-ed43bb633465 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:35.382638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-c5c38d7c-8f7a-46d2-a8a9-2a272aa5cf01 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:35.382789 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 88/178] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:35.382916 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c5c38d7c-8f7a-46d2-a8a9-2a272aa5cf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:35.389927 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b37a0-2528-4311-ba0f-ed43bb633465 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:35.492030 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 91/179] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:35.493181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c38d7c-8f7a-46d2-a8a9-2a272aa5cf01 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:35.505776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:35.561418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b37a0-2528-4311-ba0f-ed43bb633465 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:35.603675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd728e6-c178-44b2-9b41-75053887460f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:35.604965 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd728e6-c178-44b2-9b41-75053887460f None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:35.605411 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd728e6-c178-44b2-9b41-75053887460f None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:35.605824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd728e6-c178-44b2-9b41-75053887460f None admin] PATH_INFO: `/v3/users/69c9c03a92b84438b307c25afc4d0f61` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:35.629311 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd728e6-c178-44b2-9b41-75053887460f None admin] RBAC: Authorizing `identity:get_user(user_id=69c9c03a92b84438b307c25afc4d0f61)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:35.631389 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd728e6-c178-44b2-9b41-75053887460f None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:35.633615 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 89/180] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:35 2022] GET /identity/v3/users/69c9c03a92b84438b307c25afc4d0f61 => generated 283 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:35.635348 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0bb40-daf7-4b2f-b2d8-d36dde6a7348 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:35.635805 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0bb40-daf7-4b2f-b2d8-d36dde6a7348 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:35.636240 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0bb40-daf7-4b2f-b2d8-d36dde6a7348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:35.660243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-0ef0bb40-daf7-4b2f-b2d8-d36dde6a7348 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:35.661820 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-0ef0bb40-daf7-4b2f-b2d8-d36dde6a7348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:35.682433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:35.693826 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c38d7c-8f7a-46d2-a8a9-2a272aa5cf01 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:35.741213 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2fab7-20c6-45cc-90cd-c2a226bb2d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:35.742545 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2fab7-20c6-45cc-90cd-c2a226bb2d6a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:35.742968 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2fab7-20c6-45cc-90cd-c2a226bb2d6a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:35.743373 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2fab7-20c6-45cc-90cd-c2a226bb2d6a None admin] PATH_INFO: `/v3/projects/9799d178c2a042baa9963766f953c6a6` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:35.745220 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db2fab7-20c6-45cc-90cd-c2a226bb2d6a None admin] RBAC: Authorizing `identity:get_project(project_id=9799d178c2a042baa9963766f953c6a6)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:35.747237 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db2fab7-20c6-45cc-90cd-c2a226bb2d6a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:35.754588 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 92/181] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:35 2022] GET /identity/v3/projects/9799d178c2a042baa9963766f953c6a6 => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:35.756638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2fab7-20c6-45cc-90cd-c2a226bb2d6a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:35.787426 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 90/182] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:35.787426 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0bb40-daf7-4b2f-b2d8-d36dde6a7348 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:35.882941 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:35.940831 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2fab7-20c6-45cc-90cd-c2a226bb2d6a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:35.979170 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814aabe-2a6a-4c35-9076-d1e8c54005a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:35.979986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814aabe-2a6a-4c35-9076-d1e8c54005a0 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:35.980267 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814aabe-2a6a-4c35-9076-d1e8c54005a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:35.980540 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814aabe-2a6a-4c35-9076-d1e8c54005a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:35.990462 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0bb40-daf7-4b2f-b2d8-d36dde6a7348 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:35.992836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814aabe-2a6a-4c35-9076-d1e8c54005a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:36.000792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-d814aabe-2a6a-4c35-9076-d1e8c54005a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:40:36.000792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:36.000792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:36.000792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:36.000792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:36.000792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:36.000792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:36.000792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:36.000792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:36.000792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:36.000792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:36.000792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:36.000792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:36.000792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:36.000792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:36.000792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:36.001832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:36.001832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:36.001832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:36.001832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:36.001832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:36.001832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:36.001832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:36.001832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:36.001832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:36.001832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:36.001832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:36.001832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:40:36.001832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:36.001832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:36.001832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:36.001832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:36.003208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:36.003208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:36.003208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:36.003208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:36.003208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:36.003208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:36.003208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:36.003208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:36.003208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:36.003208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:36.003208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:40:36.003208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:40:36.003208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 93/183] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:36.003208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814aabe-2a6a-4c35-9076-d1e8c54005a0 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:36.035037 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:36.055110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5fd0b-69b3-45b8-a61b-b80b5acdd6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:36.055945 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5fd0b-69b3-45b8-a61b-b80b5acdd6d1 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:36.056195 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5fd0b-69b3-45b8-a61b-b80b5acdd6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:36.056869 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5fd0b-69b3-45b8-a61b-b80b5acdd6d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:36.068346 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5fd0b-69b3-45b8-a61b-b80b5acdd6d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:36.076225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-eef5fd0b-69b3-45b8-a61b-b80b5acdd6d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:40:36.076225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:36.076225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:36.076225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:36.076225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:36.076225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:36.076225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:36.076225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:36.076225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:36.076225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:36.076225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:36.076225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:36.076225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:36.076225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:36.076225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:36.076225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:36.077944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:36.077944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:36.077944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:36.077944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:36.077944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:36.077944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:36.077944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:36.077944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:36.077944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:36.077944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:36.077944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:36.077944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:40:36.077944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:36.077944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:36.077944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:36.077944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:36.079933 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:36.079933 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:36.079933 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:36.079933 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:36.079933 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:36.079933 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:36.079933 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:36.079933 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:36.079933 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:36.079933 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:36.079933 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:40:36.079933 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:40:36.079933 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 91/184] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:40:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:36.079933 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5fd0b-69b3-45b8-a61b-b80b5acdd6d1 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:36.152518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814aabe-2a6a-4c35-9076-d1e8c54005a0 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:36.196089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5fd0b-69b3-45b8-a61b-b80b5acdd6d1 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:36.198616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc81c4-67f2-4af4-aa62-ba7f8b7d95b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:36.200332 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc81c4-67f2-4af4-aa62-ba7f8b7d95b5 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:36.200702 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc81c4-67f2-4af4-aa62-ba7f8b7d95b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:36.201062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc81c4-67f2-4af4-aa62-ba7f8b7d95b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:36.204565 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:36.239967 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a57ed2-3103-4bd6-8a8a-2e320f0cb2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:36.240653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a57ed2-3103-4bd6-8a8a-2e320f0cb2e7 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:36.240931 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a57ed2-3103-4bd6-8a8a-2e320f0cb2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:36.241197 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a57ed2-3103-4bd6-8a8a-2e320f0cb2e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:36.242113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a57ed2-3103-4bd6-8a8a-2e320f0cb2e7 None admin] RBAC: Adding query filter params (role.id=d451467ccbbf4149abbfc9b0f76b9339, scope.project.id=9799d178c2a042baa9963766f953c6a6, user.id=74e87dd6b03d4b6ca41070df6e3f2f11) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:36.242427 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a57ed2-3103-4bd6-8a8a-2e320f0cb2e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:36.243981 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a57ed2-3103-4bd6-8a8a-2e320f0cb2e7 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:36.251324 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 92/185] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:40:36 2022] GET /identity/v3/role_assignments?user.id=74e87dd6b03d4b6ca41070df6e3f2f11&scope.project.id=9799d178c2a042baa9963766f953c6a6&role.id=d451467ccbbf4149abbfc9b0f76b9339 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:36.252987 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc75c79-3c5d-4b29-b28e-2ed8f791d240 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:36.253393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc75c79-3c5d-4b29-b28e-2ed8f791d240 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:36.253804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc75c79-3c5d-4b29-b28e-2ed8f791d240 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:36.258182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 93/186] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:36.258182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc75c79-3c5d-4b29-b28e-2ed8f791d240 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:36.330916 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc81c4-67f2-4af4-aa62-ba7f8b7d95b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:36.334057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc81c4-67f2-4af4-aa62-ba7f8b7d95b5 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:36.385083 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:36.446374 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc75c79-3c5d-4b29-b28e-2ed8f791d240 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:36.494236 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 94/187] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:40:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:40:36.495841 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc81c4-67f2-4af4-aa62-ba7f8b7d95b5 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:36.517890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ef470-27ee-4aa7-a139-953b36c6e339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:36.519741 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ef470-27ee-4aa7-a139-953b36c6e339 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:36.520247 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ef470-27ee-4aa7-a139-953b36c6e339 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:36.520743 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ef470-27ee-4aa7-a139-953b36c6e339 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:36.522729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ef470-27ee-4aa7-a139-953b36c6e339 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:36.524226 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ef470-27ee-4aa7-a139-953b36c6e339 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:36.525957 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 94/188] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:40:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:36.527634 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ef470-27ee-4aa7-a139-953b36c6e339 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:36.583313 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:36.671476 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc81c4-67f2-4af4-aa62-ba7f8b7d95b5 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:36.671755 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ef470-27ee-4aa7-a139-953b36c6e339 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:36.700751 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:36.715404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2247074-36d0-4a29-9a59-48f48ed6f58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:36.716432 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2247074-36d0-4a29-9a59-48f48ed6f58d None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:36.716871 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2247074-36d0-4a29-9a59-48f48ed6f58d None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:36.717304 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2247074-36d0-4a29-9a59-48f48ed6f58d None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:36.718711 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2247074-36d0-4a29-9a59-48f48ed6f58d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:36.719232 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2247074-36d0-4a29-9a59-48f48ed6f58d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:36.721722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2247074-36d0-4a29-9a59-48f48ed6f58d None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:36.722363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b56e34-d31f-4610-89d4-6c3a4ef51b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:36.723460 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b56e34-d31f-4610-89d4-6c3a4ef51b74 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:36.723909 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b56e34-d31f-4610-89d4-6c3a4ef51b74 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:36.724327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b56e34-d31f-4610-89d4-6c3a4ef51b74 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:36.726068 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b56e34-d31f-4610-89d4-6c3a4ef51b74 None admin] RBAC: Adding query filter params (name=member) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:36.726068 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b56e34-d31f-4610-89d4-6c3a4ef51b74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:36.728582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b56e34-d31f-4610-89d4-6c3a4ef51b74 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:36.735284 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 95/189] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:36.737425 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f45548-bd05-45b4-a031-c0dc2dafcab2 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:36.737741 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f45548-bd05-45b4-a031-c0dc2dafcab2 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:36.738005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f45548-bd05-45b4-a031-c0dc2dafcab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:36.745975 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 95/190] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:40:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:36.747099 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b56e34-d31f-4610-89d4-6c3a4ef51b74 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:36.762253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-81f45548-bd05-45b4-a031-c0dc2dafcab2 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:36.764730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-81f45548-bd05-45b4-a031-c0dc2dafcab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:36.865539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 96/191] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:36.867011 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f45548-bd05-45b4-a031-c0dc2dafcab2 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:36.898983 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b56e34-d31f-4610-89d4-6c3a4ef51b74 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:36.929535 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:36.939984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864b614-7f67-41a3-9436-13d48290d9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:36.940888 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864b614-7f67-41a3-9436-13d48290d9fc None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:36.941165 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864b614-7f67-41a3-9436-13d48290d9fc None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:36.941443 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864b614-7f67-41a3-9436-13d48290d9fc None admin] PATH_INFO: `/v3/projects/212e33252b164b33ae38cf0925858a09` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:36.942737 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864b614-7f67-41a3-9436-13d48290d9fc None admin] RBAC: Authorizing `identity:get_project(project_id=212e33252b164b33ae38cf0925858a09)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:36.944218 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864b614-7f67-41a3-9436-13d48290d9fc None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:36.945979 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 96/192] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:36 2022] GET /identity/v3/projects/212e33252b164b33ae38cf0925858a09 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:36.946806 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864b614-7f67-41a3-9436-13d48290d9fc None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:37.012866 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:37.037157 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f45548-bd05-45b4-a031-c0dc2dafcab2 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:37.084904 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864b614-7f67-41a3-9436-13d48290d9fc None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:37.090417 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d001e6-2fb6-4bb3-9914-80e8478ecccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:37.091346 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d001e6-2fb6-4bb3-9914-80e8478ecccc None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:37.091729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d001e6-2fb6-4bb3-9914-80e8478ecccc None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:37.092122 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d001e6-2fb6-4bb3-9914-80e8478ecccc None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:37.092967 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d001e6-2fb6-4bb3-9914-80e8478ecccc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:37.094964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d001e6-2fb6-4bb3-9914-80e8478ecccc None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:37.120941 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 97/193] 10.222.0.78 () {60 vars in 1132 bytes} [Wed May 25 14:40:36 2022] POST /identity/v3/users => generated 297 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:37.121900 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d001e6-2fb6-4bb3-9914-80e8478ecccc None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:37.137306 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6a60c-ac7b-4b62-9885-e2aa488faf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:37.138564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6a60c-ac7b-4b62-9885-e2aa488faf7e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:37.139033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6a60c-ac7b-4b62-9885-e2aa488faf7e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:37.139512 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6a60c-ac7b-4b62-9885-e2aa488faf7e None admin] PATH_INFO: `/v3/users/69c9c03a92b84438b307c25afc4d0f61` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:37.141828 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6a60c-ac7b-4b62-9885-e2aa488faf7e None admin] RBAC: Authorizing `identity:get_user(user_id=69c9c03a92b84438b307c25afc4d0f61)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:37.145023 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6a60c-ac7b-4b62-9885-e2aa488faf7e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:37.155203 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 97/194] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:36 2022] GET /identity/v3/users/69c9c03a92b84438b307c25afc4d0f61 => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:37.155203 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6a60c-ac7b-4b62-9885-e2aa488faf7e None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:37.300370 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:37.329781 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d001e6-2fb6-4bb3-9914-80e8478ecccc None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:37.343790 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6a60c-ac7b-4b62-9885-e2aa488faf7e None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:37.381363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:37.396571 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2b9a3-8241-4a36-a07f-e1609e500e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:37.398634 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2b9a3-8241-4a36-a07f-e1609e500e76 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:37.399119 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2b9a3-8241-4a36-a07f-e1609e500e76 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:37.399730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2b9a3-8241-4a36-a07f-e1609e500e76 None admin] PATH_INFO: `/v3/users/74e87dd6b03d4b6ca41070df6e3f2f11` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:37.401918 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2b9a3-8241-4a36-a07f-e1609e500e76 None admin] RBAC: Authorizing `identity:get_user(user_id=74e87dd6b03d4b6ca41070df6e3f2f11)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:37.404763 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2b9a3-8241-4a36-a07f-e1609e500e76 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:37.408580 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 98/195] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:37 2022] GET /identity/v3/users/74e87dd6b03d4b6ca41070df6e3f2f11 => generated 275 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:37.409377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabdbb0-573c-4272-9106-f8bedf648b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:37.410213 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2b9a3-8241-4a36-a07f-e1609e500e76 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:37.410965 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabdbb0-573c-4272-9106-f8bedf648b28 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:37.411469 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabdbb0-573c-4272-9106-f8bedf648b28 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:37.411889 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabdbb0-573c-4272-9106-f8bedf648b28 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:37.432792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabdbb0-573c-4272-9106-f8bedf648b28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:37.445076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-2dabdbb0-573c-4272-9106-f8bedf648b28 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:40:37.445076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:37.445076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:37.445076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:37.445076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:37.445076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:37.445076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:37.445076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:37.445076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:37.445076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:37.445076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:37.445076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:37.445076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:37.445076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:37.445076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:37.445076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:37.446372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:37.446372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:37.446372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:37.446372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:37.446372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:37.446372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:37.446372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:37.446372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:37.446372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:37.446372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:37.446372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:37.446372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:40:37.446372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:37.446372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:37.446372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:37.446372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:37.447201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:37.447201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:37.447201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:37.447201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:37.447201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:37.447201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:37.447201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:37.447201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:37.447201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:37.447201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:37.447201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:40:37.447201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:40:37.447853 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 98/196] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:37.448924 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabdbb0-573c-4272-9106-f8bedf648b28 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:37.601273 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2b9a3-8241-4a36-a07f-e1609e500e76 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:37.673592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e31256-e703-4e9b-914f-d67704dd40e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:37.674927 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e31256-e703-4e9b-914f-d67704dd40e7 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:37.675399 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e31256-e703-4e9b-914f-d67704dd40e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:37.675882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e31256-e703-4e9b-914f-d67704dd40e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:37.678069 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e31256-e703-4e9b-914f-d67704dd40e7 None admin] RBAC: Adding query filter params (role.id=d451467ccbbf4149abbfc9b0f76b9339, scope.project.id=212e33252b164b33ae38cf0925858a09, user.id=69c9c03a92b84438b307c25afc4d0f61) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:37.678577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e31256-e703-4e9b-914f-d67704dd40e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:37.682615 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e31256-e703-4e9b-914f-d67704dd40e7 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:37.692207 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabdbb0-573c-4272-9106-f8bedf648b28 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:37.695483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 99/197] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:40:37 2022] GET /identity/v3/role_assignments?user.id=69c9c03a92b84438b307c25afc4d0f61&scope.project.id=212e33252b164b33ae38cf0925858a09&role.id=d451467ccbbf4149abbfc9b0f76b9339 => generated 259 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:37.697530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e31256-e703-4e9b-914f-d67704dd40e7 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:37.741819 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:37.855451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13d610-f51e-4445-a331-c0807ecf0bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:37.856532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13d610-f51e-4445-a331-c0807ecf0bb0 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:37.857040 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13d610-f51e-4445-a331-c0807ecf0bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:37.857547 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13d610-f51e-4445-a331-c0807ecf0bb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:37.934049 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e31256-e703-4e9b-914f-d67704dd40e7 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:38.013041 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44e0ce-2b4e-4703-8aca-a48afdd268c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:38.014207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44e0ce-2b4e-4703-8aca-a48afdd268c3 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:38.014753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44e0ce-2b4e-4703-8aca-a48afdd268c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:38.015662 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44e0ce-2b4e-4703-8aca-a48afdd268c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:38.029993 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:38.087685 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13d610-f51e-4445-a331-c0807ecf0bb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:38.095735 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13d610-f51e-4445-a331-c0807ecf0bb0 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:38.150435 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44e0ce-2b4e-4703-8aca-a48afdd268c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:38.152490 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44e0ce-2b4e-4703-8aca-a48afdd268c3 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:38.181669 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:38.292231 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 100/198] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:40:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:40:38.293045 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44e0ce-2b4e-4703-8aca-a48afdd268c3 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:38.405491 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 99/199] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:40:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 957 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:40:38.406463 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13d610-f51e-4445-a331-c0807ecf0bb0 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:38.425210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44e0ce-2b4e-4703-8aca-a48afdd268c3 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:38.462637 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:38.470368 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3abd0-d869-4bbd-b234-682e58a86601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:38.471922 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3abd0-d869-4bbd-b234-682e58a86601 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:38.472209 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3abd0-d869-4bbd-b234-682e58a86601 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:38.472482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3abd0-d869-4bbd-b234-682e58a86601 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:38.473260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3abd0-d869-4bbd-b234-682e58a86601 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:38.473565 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3abd0-d869-4bbd-b234-682e58a86601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:38.475153 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3abd0-d869-4bbd-b234-682e58a86601 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:38.485912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 101/200] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:40:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:38.491778 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3abd0-d869-4bbd-b234-682e58a86601 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:38.631792 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3abd0-d869-4bbd-b234-682e58a86601 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:38.688391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:38.710996 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491e595-60ab-4c84-95f6-a24933051289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:38.711894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491e595-60ab-4c84-95f6-a24933051289 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:38.712175 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491e595-60ab-4c84-95f6-a24933051289 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:38.712463 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491e595-60ab-4c84-95f6-a24933051289 None admin] PATH_INFO: `/v3/projects/212e33252b164b33ae38cf0925858a09` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:38.715018 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491e595-60ab-4c84-95f6-a24933051289 None admin] RBAC: Authorizing `identity:get_project(project_id=212e33252b164b33ae38cf0925858a09)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:38.716596 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491e595-60ab-4c84-95f6-a24933051289 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:38.728353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 102/201] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:38 2022] GET /identity/v3/projects/212e33252b164b33ae38cf0925858a09 => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:38.729407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491e595-60ab-4c84-95f6-a24933051289 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:38.739113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13d610-f51e-4445-a331-c0807ecf0bb0 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:38.783164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d73306-7cbf-48f8-99df-766fc01d1033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:38.785428 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d73306-7cbf-48f8-99df-766fc01d1033 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:38.785857 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d73306-7cbf-48f8-99df-766fc01d1033 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:38.786332 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d73306-7cbf-48f8-99df-766fc01d1033 None admin] PATH_INFO: `/v3/projects/39ae31a66cc64d0095d6f865e887f407` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:38.797220 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d73306-7cbf-48f8-99df-766fc01d1033 None admin] RBAC: Authorizing `identity:get_project(project_id=39ae31a66cc64d0095d6f865e887f407)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:38.804126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d73306-7cbf-48f8-99df-766fc01d1033 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:38.817800 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 100/202] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:38 2022] GET /identity/v3/projects/39ae31a66cc64d0095d6f865e887f407 => generated 312 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:38.820780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d73306-7cbf-48f8-99df-766fc01d1033 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:38.929920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:38.998471 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491e595-60ab-4c84-95f6-a24933051289 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:39.067950 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c118f0f-7076-4e53-b574-fa509af09376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:39.069964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c118f0f-7076-4e53-b574-fa509af09376 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:39.070756 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c118f0f-7076-4e53-b574-fa509af09376 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:39.071443 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c118f0f-7076-4e53-b574-fa509af09376 None admin] PATH_INFO: `/v3/users/b3413b1ed0e54a73834c597e8e23a440` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:39.074198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c118f0f-7076-4e53-b574-fa509af09376 None admin] RBAC: Authorizing `identity:get_user(user_id=b3413b1ed0e54a73834c597e8e23a440)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:39.077373 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c118f0f-7076-4e53-b574-fa509af09376 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:39.082058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 103/203] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:38 2022] GET /identity/v3/users/b3413b1ed0e54a73834c597e8e23a440 => generated 289 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:39.083902 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c118f0f-7076-4e53-b574-fa509af09376 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:39.223241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d73306-7cbf-48f8-99df-766fc01d1033 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:39.293723 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c118f0f-7076-4e53-b574-fa509af09376 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:39.326632 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:39.359856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4bf3c-cf9e-439a-adfd-7b00db06e4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:39.361221 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4bf3c-cf9e-439a-adfd-7b00db06e4e8 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:39.361976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4bf3c-cf9e-439a-adfd-7b00db06e4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:39.362791 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4bf3c-cf9e-439a-adfd-7b00db06e4e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:39.364769 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4bf3c-cf9e-439a-adfd-7b00db06e4e8 None admin] RBAC: Adding query filter params (role.id=a3da3c62738d4790b9dac512b1568b52, scope.project.id=39ae31a66cc64d0095d6f865e887f407, user.id=74e87dd6b03d4b6ca41070df6e3f2f11) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:39.365653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4bf3c-cf9e-439a-adfd-7b00db06e4e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:39.368669 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4bf3c-cf9e-439a-adfd-7b00db06e4e8 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:39.375243 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d508f1-5fc5-4859-9e70-27ab05edd5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:39.376406 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d508f1-5fc5-4859-9e70-27ab05edd5f2 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:39.376920 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d508f1-5fc5-4859-9e70-27ab05edd5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:39.378050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d508f1-5fc5-4859-9e70-27ab05edd5f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:39.379717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d508f1-5fc5-4859-9e70-27ab05edd5f2 None admin] RBAC: Adding query filter params (role.id=e566b37cd92f467198b5dc40f183effc, scope.project.id=212e33252b164b33ae38cf0925858a09, user.id=69c9c03a92b84438b307c25afc4d0f61) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:39.380290 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d508f1-5fc5-4859-9e70-27ab05edd5f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:39.383071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 104/204] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:40:39 2022] GET /identity/v3/role_assignments?user.id=74e87dd6b03d4b6ca41070df6e3f2f11&scope.project.id=39ae31a66cc64d0095d6f865e887f407&role.id=a3da3c62738d4790b9dac512b1568b52 => generated 608 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:39.384752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4bf3c-cf9e-439a-adfd-7b00db06e4e8 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:39.391681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d508f1-5fc5-4859-9e70-27ab05edd5f2 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:39.406567 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 101/205] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:40:38 2022] GET /identity/v3/role_assignments?user.id=69c9c03a92b84438b307c25afc4d0f61&scope.project.id=212e33252b164b33ae38cf0925858a09&role.id=e566b37cd92f467198b5dc40f183effc => generated 259 bytes in 587 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:39.581182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4bf3c-cf9e-439a-adfd-7b00db06e4e8 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:39.624005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b7f30-a06b-4096-9a78-81655c01550a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:39.625251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b7f30-a06b-4096-9a78-81655c01550a None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:39.625751 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b7f30-a06b-4096-9a78-81655c01550a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:39.626280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b7f30-a06b-4096-9a78-81655c01550a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:39.696336 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:39.720925 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b7f30-a06b-4096-9a78-81655c01550a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:39.722561 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b7f30-a06b-4096-9a78-81655c01550a None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:39.851120 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 105/206] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:40:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:40:39.857275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d508f1-5fc5-4859-9e70-27ab05edd5f2 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:39.878392 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:40.014774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d508f1-5fc5-4859-9e70-27ab05edd5f2 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:40.070189 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430a1d9-f296-43e6-88e9-44763d6ac8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:40.071700 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430a1d9-f296-43e6-88e9-44763d6ac8f0 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:40.072213 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430a1d9-f296-43e6-88e9-44763d6ac8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:40.072712 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430a1d9-f296-43e6-88e9-44763d6ac8f0 None admin] PATH_INFO: `/v3/projects/9799d178c2a042baa9963766f953c6a6` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:40.075152 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7430a1d9-f296-43e6-88e9-44763d6ac8f0 None admin] RBAC: Authorizing `identity:get_project(project_id=9799d178c2a042baa9963766f953c6a6)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:40.077965 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7430a1d9-f296-43e6-88e9-44763d6ac8f0 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:40.081041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 102/207] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:39 2022] GET /identity/v3/projects/9799d178c2a042baa9963766f953c6a6 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:40.089204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b7f30-a06b-4096-9a78-81655c01550a None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:40.245450 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:40.282262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b7f30-a06b-4096-9a78-81655c01550a None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:40.351335 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af36c3-b8eb-431c-aa9c-b0074d1a13ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:40.352586 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af36c3-b8eb-431c-aa9c-b0074d1a13ca None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:40.353074 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af36c3-b8eb-431c-aa9c-b0074d1a13ca None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:40.353576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af36c3-b8eb-431c-aa9c-b0074d1a13ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:40.355250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af36c3-b8eb-431c-aa9c-b0074d1a13ca None admin] RBAC: Adding query filter params (role.id=ebb0d7a0f606450a8dbfee510b7b0b80, scope.project.id=9799d178c2a042baa9963766f953c6a6, user.id=b3413b1ed0e54a73834c597e8e23a440) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:40.355804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af36c3-b8eb-431c-aa9c-b0074d1a13ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:40.358655 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af36c3-b8eb-431c-aa9c-b0074d1a13ca None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:40.370825 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 106/208] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:40:40 2022] GET /identity/v3/role_assignments?user.id=b3413b1ed0e54a73834c597e8e23a440&scope.project.id=9799d178c2a042baa9963766f953c6a6&role.id=ebb0d7a0f606450a8dbfee510b7b0b80 => generated 608 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:40.694055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff89eb-53b2-4d67-9c21-ae6239f7672c None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:40.695043 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff89eb-53b2-4d67-9c21-ae6239f7672c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:40.695577 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff89eb-53b2-4d67-9c21-ae6239f7672c None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:40.697309 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 103/209] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:40.703779 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e0b15-6d25-4d88-90b6-0d2ad3bbe0bf None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:40.704423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e0b15-6d25-4d88-90b6-0d2ad3bbe0bf None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:40.704908 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e0b15-6d25-4d88-90b6-0d2ad3bbe0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:40.726528 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-ce9e0b15-6d25-4d88-90b6-0d2ad3bbe0bf None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:40.728156 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-ce9e0b15-6d25-4d88-90b6-0d2ad3bbe0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:40.825632 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 107/210] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:40.851801 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff89eb-53b2-4d67-9c21-ae6239f7672c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:40.988363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:41.023471 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff89eb-53b2-4d67-9c21-ae6239f7672c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:41.028678 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25caaa6-4103-41dc-8a76-cc4e0ec5b0d5 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:41.029296 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25caaa6-4103-41dc-8a76-cc4e0ec5b0d5 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:41.029774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25caaa6-4103-41dc-8a76-cc4e0ec5b0d5 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:41.031511 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 108/211] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:40:41.042443 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882487b1-77a3-4fe4-9573-2c7e59bd4972 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:41.043437 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882487b1-77a3-4fe4-9573-2c7e59bd4972 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:41.044182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882487b1-77a3-4fe4-9573-2c7e59bd4972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:41.081987 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-882487b1-77a3-4fe4-9573-2c7e59bd4972 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:41.084576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-882487b1-77a3-4fe4-9573-2c7e59bd4972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:41.091639 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54b7ae-03c9-4a5b-94f0-d3f7e62cd126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:41.092836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54b7ae-03c9-4a5b-94f0-d3f7e62cd126 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:41.093237 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54b7ae-03c9-4a5b-94f0-d3f7e62cd126 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:41.093631 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54b7ae-03c9-4a5b-94f0-d3f7e62cd126 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:41.095385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d54b7ae-03c9-4a5b-94f0-d3f7e62cd126 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:41.097589 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d54b7ae-03c9-4a5b-94f0-d3f7e62cd126 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:41.100051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 104/212] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:40:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:41.107526 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54b7ae-03c9-4a5b-94f0-d3f7e62cd126 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:41.224564 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 109/213] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:41.265366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882487b1-77a3-4fe4-9573-2c7e59bd4972 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:41.272029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54b7ae-03c9-4a5b-94f0-d3f7e62cd126 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:41.285124 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:41.338456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004e363-adc6-4449-a194-38771305ea59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:41.339678 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004e363-adc6-4449-a194-38771305ea59 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:41.340193 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004e363-adc6-4449-a194-38771305ea59 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:41.340702 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004e363-adc6-4449-a194-38771305ea59 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:41.342170 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004e363-adc6-4449-a194-38771305ea59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:41.346565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004e363-adc6-4449-a194-38771305ea59 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:41.396967 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 105/214] 10.222.0.78 () {60 vars in 1132 bytes} [Wed May 25 14:40:41 2022] POST /identity/v3/users => generated 297 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:40:41.457530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882487b1-77a3-4fe4-9573-2c7e59bd4972 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:41.499401 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a369153-5eb9-4c44-ac6c-a5907094a117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:41.500223 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a369153-5eb9-4c44-ac6c-a5907094a117 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:41.500495 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a369153-5eb9-4c44-ac6c-a5907094a117 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:41.500774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a369153-5eb9-4c44-ac6c-a5907094a117 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:41.513663 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a369153-5eb9-4c44-ac6c-a5907094a117 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:41.521882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-7a369153-5eb9-4c44-ac6c-a5907094a117 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:40:41.521882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:41.521882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:41.521882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:41.521882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:41.521882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:41.521882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:41.521882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:41.521882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:41.521882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:41.521882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:41.521882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:41.521882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:41.521882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:41.521882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:41.521882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:41.523519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:41.523519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:41.523519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:41.523519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:41.523519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:41.523519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:41.523519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:41.523519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:41.523519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:41.523519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:41.523519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:41.523519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:40:41.523519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:41.523519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:41.523519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:41.523519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:41.525539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:41.525539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:41.525539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:41.525539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:41.525539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:41.525539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:41.525539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:41.525539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:41.525539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:41.525539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:41.525539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:40:41.525539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:40:41.525539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 110/215] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:40:41.532965 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004e363-adc6-4449-a194-38771305ea59 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:41.559414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:41.769285 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004e363-adc6-4449-a194-38771305ea59 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:41.848058 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03c528-4b00-401b-890b-9ceae8877ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:41.849166 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03c528-4b00-401b-890b-9ceae8877ba8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:41.849735 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03c528-4b00-401b-890b-9ceae8877ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:41.850260 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03c528-4b00-401b-890b-9ceae8877ba8 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:41.851806 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03c528-4b00-401b-890b-9ceae8877ba8 None admin] RBAC: Adding query filter params (name=member) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:41.852362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03c528-4b00-401b-890b-9ceae8877ba8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:41.855194 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03c528-4b00-401b-890b-9ceae8877ba8 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:41.876409 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 106/216] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:40:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:41.883420 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a369153-5eb9-4c44-ac6c-a5907094a117 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:42.007170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:42.052158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a369153-5eb9-4c44-ac6c-a5907094a117 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:42.091318 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3b493-7bb6-4fec-bc7e-36bf8b5a499c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:42.092230 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3b493-7bb6-4fec-bc7e-36bf8b5a499c None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:42.092515 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3b493-7bb6-4fec-bc7e-36bf8b5a499c None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:42.092814 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3b493-7bb6-4fec-bc7e-36bf8b5a499c None admin] PATH_INFO: `/v3/users/9b3baedf44cb47318665288b67ae20d0` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:42.109653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b3b493-7bb6-4fec-bc7e-36bf8b5a499c None admin] RBAC: Authorizing `identity:get_user(user_id=9b3baedf44cb47318665288b67ae20d0)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:42.111088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b3b493-7bb6-4fec-bc7e-36bf8b5a499c None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:42.113321 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 111/217] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:41 2022] GET /identity/v3/users/9b3baedf44cb47318665288b67ae20d0 => generated 297 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:42.123931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03c528-4b00-401b-890b-9ceae8877ba8 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:42.214898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf081f23-2e29-4105-a839-22c938014c56 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:42.215355 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf081f23-2e29-4105-a839-22c938014c56 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:42.215739 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf081f23-2e29-4105-a839-22c938014c56 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:42.217588 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 112/218] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:42.238990 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29004e84-dc80-4dac-87c9-096800e60abc None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:42.239880 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29004e84-dc80-4dac-87c9-096800e60abc None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:42.239880 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29004e84-dc80-4dac-87c9-096800e60abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:42.270464 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-29004e84-dc80-4dac-87c9-096800e60abc None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:42.271926 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-29004e84-dc80-4dac-87c9-096800e60abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:42.317088 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:42.421392 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03c528-4b00-401b-890b-9ceae8877ba8 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:42.472701 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 113/219] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:40:42.497396 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29004e84-dc80-4dac-87c9-096800e60abc None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:42.504262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be1b24-0754-4748-9d67-b70ca531f61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:42.505379 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be1b24-0754-4748-9d67-b70ca531f61c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:42.505893 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be1b24-0754-4748-9d67-b70ca531f61c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:42.506432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be1b24-0754-4748-9d67-b70ca531f61c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:42.570896 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:42.627477 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29004e84-dc80-4dac-87c9-096800e60abc None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:42.671361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d4b67-2b96-479b-a667-8eebb24a7e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:42.672671 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d4b67-2b96-479b-a667-8eebb24a7e9f None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:42.673019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d4b67-2b96-479b-a667-8eebb24a7e9f None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:42.673451 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d4b67-2b96-479b-a667-8eebb24a7e9f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:42.689592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466d4b67-2b96-479b-a667-8eebb24a7e9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:42.699216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-466d4b67-2b96-479b-a667-8eebb24a7e9f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:40:42.699216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:42.699216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:42.699216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:42.699216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:42.699216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:42.699216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:42.699216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:42.699216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:42.699216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:42.699216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:42.699216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:42.699216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:42.699216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:42.699216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:42.699216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:42.700717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:42.700717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:42.700717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:42.700717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:42.700717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:42.700717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:42.700717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:42.700717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:42.700717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:42.700717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:42.700717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:42.700717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:40:42.700717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:42.700717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:42.700717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:42.700717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:42.701964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:42.701964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:42.701964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:42.701964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:42.701964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:42.701964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:42.701964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:42.701964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:42.701964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:42.701964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:42.701964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:40:42.701964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:40:42.701964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 114/220] 10.222.0.78 () {56 vars in 1114 bytes} [Wed May 25 14:40:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:40:42.708025 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d4b67-2b96-479b-a667-8eebb24a7e9f None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:42.720199 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be1b24-0754-4748-9d67-b70ca531f61c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:42.721889 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be1b24-0754-4748-9d67-b70ca531f61c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:42.808774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:42.888054 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 107/221] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:40:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 765 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:40:42.898408 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be1b24-0754-4748-9d67-b70ca531f61c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:43.037463 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be1b24-0754-4748-9d67-b70ca531f61c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:43.042227 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d4b67-2b96-479b-a667-8eebb24a7e9f None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:43.076501 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:43.114675 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825951c-5b92-4e7c-992a-4eccd9ea07ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:43.115571 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825951c-5b92-4e7c-992a-4eccd9ea07ec None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:43.115859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825951c-5b92-4e7c-992a-4eccd9ea07ec None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:43.116142 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825951c-5b92-4e7c-992a-4eccd9ea07ec None admin] PATH_INFO: `/v3/projects/212e33252b164b33ae38cf0925858a09` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:43.119362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d825951c-5b92-4e7c-992a-4eccd9ea07ec None admin] RBAC: Authorizing `identity:get_project(project_id=212e33252b164b33ae38cf0925858a09)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:43.120903 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d825951c-5b92-4e7c-992a-4eccd9ea07ec None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:43.123007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 108/222] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:42 2022] GET /identity/v3/projects/212e33252b164b33ae38cf0925858a09 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:43.126691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e0e96-bd99-4705-966e-081b75b8e4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:43.127417 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e0e96-bd99-4705-966e-081b75b8e4e4 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:43.127698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e0e96-bd99-4705-966e-081b75b8e4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:43.127970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e0e96-bd99-4705-966e-081b75b8e4e4 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:43.128748 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e0e96-bd99-4705-966e-081b75b8e4e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:43.129050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e0e96-bd99-4705-966e-081b75b8e4e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:43.130499 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e0e96-bd99-4705-966e-081b75b8e4e4 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:43.131981 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825951c-5b92-4e7c-992a-4eccd9ea07ec None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:43.151028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 115/223] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:40:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 444 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:43.159881 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e0e96-bd99-4705-966e-081b75b8e4e4 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:43.215330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:43.301430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825951c-5b92-4e7c-992a-4eccd9ea07ec None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:43.312813 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e0e96-bd99-4705-966e-081b75b8e4e4 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:43.341760 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b6298-91db-4528-a3c9-84d5388054f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:43.342877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b6298-91db-4528-a3c9-84d5388054f9 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:43.343367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b6298-91db-4528-a3c9-84d5388054f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:43.343845 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b6298-91db-4528-a3c9-84d5388054f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:43.344936 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b6298-91db-4528-a3c9-84d5388054f9 None admin] RBAC: Adding query filter params (role.id=a3da3c62738d4790b9dac512b1568b52, scope.project.id=212e33252b164b33ae38cf0925858a09, user.id=9b3baedf44cb47318665288b67ae20d0) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:43.345461 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b6298-91db-4528-a3c9-84d5388054f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:43.347273 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b6298-91db-4528-a3c9-84d5388054f9 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:43.353060 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb2858-740e-454a-861b-c648c1220b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:43.353952 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb2858-740e-454a-861b-c648c1220b88 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:43.354269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb2858-740e-454a-861b-c648c1220b88 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:43.354675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb2858-740e-454a-861b-c648c1220b88 None admin] PATH_INFO: `/v3/users/69c9c03a92b84438b307c25afc4d0f61` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:43.355117 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 109/224] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:40:43 2022] GET /identity/v3/role_assignments?user.id=9b3baedf44cb47318665288b67ae20d0&scope.project.id=212e33252b164b33ae38cf0925858a09&role.id=a3da3c62738d4790b9dac512b1568b52 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:43.355944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb2858-740e-454a-861b-c648c1220b88 None admin] RBAC: Authorizing `identity:get_user(user_id=69c9c03a92b84438b307c25afc4d0f61)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:43.356988 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200d125-e67e-4e7a-a86e-4e9512f3159b None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:43.357510 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200d125-e67e-4e7a-a86e-4e9512f3159b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:43.357624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb2858-740e-454a-861b-c648c1220b88 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:43.358157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200d125-e67e-4e7a-a86e-4e9512f3159b None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:43.359787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 110/226] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:43.364694 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 116/226] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:43 2022] GET /identity/v3/users/69c9c03a92b84438b307c25afc4d0f61 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:43.369089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb2858-740e-454a-861b-c648c1220b88 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:43.369900 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b6c80-58b7-41b8-9b60-ae9127e74db3 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:43.370465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b6c80-58b7-41b8-9b60-ae9127e74db3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:43.370943 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b6c80-58b7-41b8-9b60-ae9127e74db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:43.390140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-046b6c80-58b7-41b8-9b60-ae9127e74db3 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:43.391819 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-046b6c80-58b7-41b8-9b60-ae9127e74db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:43.462621 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:43.478969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 111/227] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:43.506554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b6c80-58b7-41b8-9b60-ae9127e74db3 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:43.543671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:43.602374 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb2858-740e-454a-861b-c648c1220b88 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:43.659931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b6c80-58b7-41b8-9b60-ae9127e74db3 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:43.703112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d69da8-df1d-4a07-acf1-09ba799c12ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:43.705865 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d69da8-df1d-4a07-acf1-09ba799c12ce None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:43.706755 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d69da8-df1d-4a07-acf1-09ba799c12ce None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:43.707524 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d69da8-df1d-4a07-acf1-09ba799c12ce None admin] PATH_INFO: `/v3/projects/212e33252b164b33ae38cf0925858a09` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:43.712125 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d69da8-df1d-4a07-acf1-09ba799c12ce None admin] RBAC: Authorizing `identity:get_project(project_id=212e33252b164b33ae38cf0925858a09)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:43.713061 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d69da8-df1d-4a07-acf1-09ba799c12ce None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:43.715369 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 117/228] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:43 2022] GET /identity/v3/projects/212e33252b164b33ae38cf0925858a09 => generated 302 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:43.727034 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d69da8-df1d-4a07-acf1-09ba799c12ce None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:43.750944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770238ea-04c4-461e-937e-909a246acefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:43.756431 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770238ea-04c4-461e-937e-909a246acefa None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:43.756431 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770238ea-04c4-461e-937e-909a246acefa None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:43.756431 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770238ea-04c4-461e-937e-909a246acefa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:43.780592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770238ea-04c4-461e-937e-909a246acefa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:43.789377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-770238ea-04c4-461e-937e-909a246acefa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:40:43.789377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:43.789377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:43.789377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:43.789377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:43.789377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:43.789377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:43.789377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:43.789377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:43.789377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:43.789377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:43.789377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:43.789377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:43.789377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:43.789377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:43.789377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:43.791056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:43.791056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:43.791056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:43.791056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:43.791056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:43.791056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:43.791056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:43.791056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:43.791056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:43.791056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:43.791056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:43.791056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:40:43.791056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:43.791056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:43.791056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:43.791056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:43.792672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:43.792672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:43.792672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:43.792672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:43.792672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:43.792672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:43.792672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:43.792672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:43.792672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:43.792672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:43.792672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:40:43.792672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:40:43.792672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 112/229] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:40:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:43.801825 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770238ea-04c4-461e-937e-909a246acefa None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:44.002577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d69da8-df1d-4a07-acf1-09ba799c12ce None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:44.030244 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:44.092908 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:44.095025 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f450d-ad14-4c25-8054-623296290b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:44.098336 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f450d-ad14-4c25-8054-623296290b12 None admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:44.099220 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f450d-ad14-4c25-8054-623296290b12 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:44.099995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f450d-ad14-4c25-8054-623296290b12 None admin] PATH_INFO: `/v3/projects/212e33252b164b33ae38cf0925858a09/users/69c9c03a92b84438b307c25afc4d0f61/roles/d451467ccbbf4149abbfc9b0f76b9339` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:44.104942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7f450d-ad14-4c25-8054-623296290b12 None admin] RBAC: Authorizing `identity:create_grant(project_id=212e33252b164b33ae38cf0925858a09, user_id=69c9c03a92b84438b307c25afc4d0f61, role_id=d451467ccbbf4149abbfc9b0f76b9339)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:44.108854 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7f450d-ad14-4c25-8054-623296290b12 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:44.139745 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 118/230] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:40:43 2022] PUT /identity/v3/projects/212e33252b164b33ae38cf0925858a09/users/69c9c03a92b84438b307c25afc4d0f61/roles/d451467ccbbf4149abbfc9b0f76b9339 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:44.152032 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770238ea-04c4-461e-937e-909a246acefa None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:44.229079 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f94382-9fca-4e13-abeb-d6e508072b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:44.229679 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f94382-9fca-4e13-abeb-d6e508072b13 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:44.229968 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f94382-9fca-4e13-abeb-d6e508072b13 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:44.230274 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f94382-9fca-4e13-abeb-d6e508072b13 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:44.231053 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f94382-9fca-4e13-abeb-d6e508072b13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:44.231364 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f94382-9fca-4e13-abeb-d6e508072b13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:44.232873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f94382-9fca-4e13-abeb-d6e508072b13 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:44.247690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 113/231] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:44.255740 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f450d-ad14-4c25-8054-623296290b12 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:44.514420 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f450d-ad14-4c25-8054-623296290b12 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:44.590057 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bb09d-b486-4e6a-8444-2559e7e00cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:44.591602 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bb09d-b486-4e6a-8444-2559e7e00cb7 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:44.592082 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bb09d-b486-4e6a-8444-2559e7e00cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:44.592582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bb09d-b486-4e6a-8444-2559e7e00cb7 None admin] PATH_INFO: `/v3/users/69c9c03a92b84438b307c25afc4d0f61` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:44.594869 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bb09d-b486-4e6a-8444-2559e7e00cb7 None admin] RBAC: Authorizing `identity:get_user(user_id=69c9c03a92b84438b307c25afc4d0f61)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:44.598140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bb09d-b486-4e6a-8444-2559e7e00cb7 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:44.601386 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 119/232] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:44 2022] GET /identity/v3/users/69c9c03a92b84438b307c25afc4d0f61 => generated 283 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:44.610542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f94382-9fca-4e13-abeb-d6e508072b13 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:44.798178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f94382-9fca-4e13-abeb-d6e508072b13 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:44.857061 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4ee45-e1e3-40f4-af39-e69b2d595333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:44.857812 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4ee45-e1e3-40f4-af39-e69b2d595333 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:44.858132 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4ee45-e1e3-40f4-af39-e69b2d595333 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:44.858427 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4ee45-e1e3-40f4-af39-e69b2d595333 None admin] PATH_INFO: `/v3/projects/212e33252b164b33ae38cf0925858a09` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:44.863092 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c4ee45-e1e3-40f4-af39-e69b2d595333 None admin] RBAC: Authorizing `identity:get_project(project_id=212e33252b164b33ae38cf0925858a09)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:44.864710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c4ee45-e1e3-40f4-af39-e69b2d595333 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:44.866977 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 114/233] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:44 2022] GET /identity/v3/projects/212e33252b164b33ae38cf0925858a09 => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:44.875758 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bb09d-b486-4e6a-8444-2559e7e00cb7 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:44.990153 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:45.122200 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bb09d-b486-4e6a-8444-2559e7e00cb7 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:45.186597 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57659e9b-56af-4d81-9e57-6e9ff5ecf2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:45.187520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57659e9b-56af-4d81-9e57-6e9ff5ecf2c8 None admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:45.187800 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57659e9b-56af-4d81-9e57-6e9ff5ecf2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:45.188076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57659e9b-56af-4d81-9e57-6e9ff5ecf2c8 None admin] PATH_INFO: `/v3/projects/212e33252b164b33ae38cf0925858a09/users/69c9c03a92b84438b307c25afc4d0f61/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:45.190570 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57659e9b-56af-4d81-9e57-6e9ff5ecf2c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=212e33252b164b33ae38cf0925858a09, user_id=69c9c03a92b84438b307c25afc4d0f61, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:45.192114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57659e9b-56af-4d81-9e57-6e9ff5ecf2c8 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:45.210338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 120/234] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:40:44 2022] PUT /identity/v3/projects/212e33252b164b33ae38cf0925858a09/users/69c9c03a92b84438b307c25afc4d0f61/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:45.361348 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb3046-81ed-48a9-bfd6-f2f65d45b86f None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:45.362612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb3046-81ed-48a9-bfd6-f2f65d45b86f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:45.362899 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb3046-81ed-48a9-bfd6-f2f65d45b86f None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:45.365742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 115/235] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:45.372158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682031ea-251a-4064-a024-8d3091e095ac None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:45.372762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682031ea-251a-4064-a024-8d3091e095ac None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:45.372762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682031ea-251a-4064-a024-8d3091e095ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:45.406706 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-682031ea-251a-4064-a024-8d3091e095ac None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:45.409382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-682031ea-251a-4064-a024-8d3091e095ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:45.565379 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 121/236] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:40:45.636020 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb3046-81ed-48a9-bfd6-f2f65d45b86f None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:45.660864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:45.800166 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb3046-81ed-48a9-bfd6-f2f65d45b86f None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:45.854064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f196bc2-98bc-4654-85aa-21d21f4d9711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:45.854882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f196bc2-98bc-4654-85aa-21d21f4d9711 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:45.855165 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f196bc2-98bc-4654-85aa-21d21f4d9711 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:45.855444 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f196bc2-98bc-4654-85aa-21d21f4d9711 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:45.879180 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f196bc2-98bc-4654-85aa-21d21f4d9711 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:45.887182 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-1f196bc2-98bc-4654-85aa-21d21f4d9711 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:40:45.887182 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:45.887182 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:45.887182 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:45.887182 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:45.887182 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:45.887182 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:45.887182 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:45.887182 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:45.887182 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:45.887182 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:45.887182 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:45.887182 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:45.887182 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:45.887182 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:45.887182 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:45.888905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:45.888905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:45.888905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:45.888905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:45.888905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:45.888905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:45.888905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:45.888905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:45.888905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:45.888905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:45.888905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:45.888905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:40:45.888905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:45.888905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:45.888905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:45.888905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:45.890484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:45.890484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:45.890484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:45.890484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:45.890484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:45.890484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:45.890484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:45.890484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:45.890484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:45.890484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:45.890484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:40:45.890484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:40:45.890484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 116/237] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:45.900733 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682031ea-251a-4064-a024-8d3091e095ac None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:45.984797 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:46.024952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ee3e9-5fe1-4b02-a6da-2be3362a761e None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:46.025605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ee3e9-5fe1-4b02-a6da-2be3362a761e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:46.026232 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ee3e9-5fe1-4b02-a6da-2be3362a761e None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:46.034114 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 117/238] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:46.036821 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582efdc4-0cda-484c-9bf5-be06f4308f64 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:46.037342 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582efdc4-0cda-484c-9bf5-be06f4308f64 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:46.037842 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582efdc4-0cda-484c-9bf5-be06f4308f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:46.091259 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-582efdc4-0cda-484c-9bf5-be06f4308f64 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:46.092583 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-582efdc4-0cda-484c-9bf5-be06f4308f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:46.133881 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682031ea-251a-4064-a024-8d3091e095ac None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:46.214071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c1ef4-7d65-4dd0-ac1c-dfc73078d43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:46.215733 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c1ef4-7d65-4dd0-ac1c-dfc73078d43e None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:46.216532 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c1ef4-7d65-4dd0-ac1c-dfc73078d43e None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:46.217659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c1ef4-7d65-4dd0-ac1c-dfc73078d43e None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:46.219649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c1ef4-7d65-4dd0-ac1c-dfc73078d43e None admin] RBAC: Adding query filter params (name=reader) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:46.220490 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c1ef4-7d65-4dd0-ac1c-dfc73078d43e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:46.224076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c1ef4-7d65-4dd0-ac1c-dfc73078d43e None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:46.240803 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 118/239] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:46.242379 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 122/240] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:40:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:46.250102 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582efdc4-0cda-484c-9bf5-be06f4308f64 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:46.274422 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:46.278741 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c1ef4-7d65-4dd0-ac1c-dfc73078d43e None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:46.461731 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582efdc4-0cda-484c-9bf5-be06f4308f64 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:46.538715 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68306c-9483-4056-9d79-ff332741c3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:46.540222 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68306c-9483-4056-9d79-ff332741c3be None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:46.540723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68306c-9483-4056-9d79-ff332741c3be None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:46.541217 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68306c-9483-4056-9d79-ff332741c3be None admin] PATH_INFO: `/v3/users/9e130e15017b49498c39de87c26d58e4` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:46.573566 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e68306c-9483-4056-9d79-ff332741c3be None admin] RBAC: Authorizing `identity:get_user(user_id=9e130e15017b49498c39de87c26d58e4)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:46.577159 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e68306c-9483-4056-9d79-ff332741c3be None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:46.581640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 119/241] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:46 2022] GET /identity/v3/users/9e130e15017b49498c39de87c26d58e4 => generated 297 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:46.592075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68306c-9483-4056-9d79-ff332741c3be None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:46.615622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:46.660232 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c1ef4-7d65-4dd0-ac1c-dfc73078d43e None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:46.741889 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:46.786918 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52687a-3ecd-47d1-887c-d7d472ef6696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:46.788633 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52687a-3ecd-47d1-887c-d7d472ef6696 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:46.793423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52687a-3ecd-47d1-887c-d7d472ef6696 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:46.793423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52687a-3ecd-47d1-887c-d7d472ef6696 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:46.793423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c52687a-3ecd-47d1-887c-d7d472ef6696 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:46.794718 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c52687a-3ecd-47d1-887c-d7d472ef6696 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:46.801976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 123/242] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:40:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 524 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:46.803591 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66dadc9-414e-4fe6-a83a-538368d68da3 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:46.803871 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66dadc9-414e-4fe6-a83a-538368d68da3 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:46.804146 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66dadc9-414e-4fe6-a83a-538368d68da3 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:46.805591 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 124/243] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:46.810824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66dadc9-414e-4fe6-a83a-538368d68da3 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:46.827067 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68306c-9483-4056-9d79-ff332741c3be None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:46.910530 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c1f92-d9b9-4f98-8844-b6d1fa63a6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:46.912199 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c1f92-d9b9-4f98-8844-b6d1fa63a6cc None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:46.913078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c1f92-d9b9-4f98-8844-b6d1fa63a6cc None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:46.913831 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c1f92-d9b9-4f98-8844-b6d1fa63a6cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:47.003214 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66dadc9-414e-4fe6-a83a-538368d68da3 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:47.035683 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c1f92-d9b9-4f98-8844-b6d1fa63a6cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:47.037319 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c1f92-d9b9-4f98-8844-b6d1fa63a6cc None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:47.058563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f23e3-bfe1-47e5-b425-dc697fe99eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:47.059148 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f23e3-bfe1-47e5-b425-dc697fe99eb2 None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:47.059424 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f23e3-bfe1-47e5-b425-dc697fe99eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:47.059691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f23e3-bfe1-47e5-b425-dc697fe99eb2 None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:47.060413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f23e3-bfe1-47e5-b425-dc697fe99eb2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:47.062266 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f23e3-bfe1-47e5-b425-dc697fe99eb2 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:47.098323 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 125/244] 10.222.0.78 () {60 vars in 1132 bytes} [Wed May 25 14:40:46 2022] POST /identity/v3/users => generated 293 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:47.099971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6026f483-027f-44a4-8de9-8d561a4ff07a None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:47.100320 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6026f483-027f-44a4-8de9-8d561a4ff07a None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:47.100584 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6026f483-027f-44a4-8de9-8d561a4ff07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:47.139779 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-6026f483-027f-44a4-8de9-8d561a4ff07a None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:47.141082 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-6026f483-027f-44a4-8de9-8d561a4ff07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:47.158634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:47.164204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:47.205887 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 120/245] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:40:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:40:47.217458 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c1f92-d9b9-4f98-8844-b6d1fa63a6cc None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:47.295841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 126/246] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:47.321044 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6026f483-027f-44a4-8de9-8d561a4ff07a None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:47.452699 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c1f92-d9b9-4f98-8844-b6d1fa63a6cc None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:47.508494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de414698-f793-45a3-ad91-437c1a21b648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:47.509505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de414698-f793-45a3-ad91-437c1a21b648 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:47.509644 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de414698-f793-45a3-ad91-437c1a21b648 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:47.509990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de414698-f793-45a3-ad91-437c1a21b648 None admin] PATH_INFO: `/v3/projects/212e33252b164b33ae38cf0925858a09` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:47.511888 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de414698-f793-45a3-ad91-437c1a21b648 None admin] RBAC: Authorizing `identity:get_project(project_id=212e33252b164b33ae38cf0925858a09)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:47.513410 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de414698-f793-45a3-ad91-437c1a21b648 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:47.517669 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 121/247] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:47 2022] GET /identity/v3/projects/212e33252b164b33ae38cf0925858a09 => generated 302 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:47.522836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de414698-f793-45a3-ad91-437c1a21b648 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:47.555334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6026f483-027f-44a4-8de9-8d561a4ff07a None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:47.586843 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:47.646908 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190813cd-4202-4a4e-b02c-4225cb98225c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:47.648295 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190813cd-4202-4a4e-b02c-4225cb98225c None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:47.648820 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190813cd-4202-4a4e-b02c-4225cb98225c None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:47.651851 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190813cd-4202-4a4e-b02c-4225cb98225c None admin] PATH_INFO: `/v3/roles/member` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:47.683564 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190813cd-4202-4a4e-b02c-4225cb98225c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:47.688217 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de414698-f793-45a3-ad91-437c1a21b648 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:47.707282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-190813cd-4202-4a4e-b02c-4225cb98225c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:40:47.707282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:47.707282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:47.707282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:47.707282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:47.707282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:47.707282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:47.707282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:47.707282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:47.707282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:47.707282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:47.707282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:47.707282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:47.707282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:47.707282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:47.707282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:47.708297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:47.708297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:47.708297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:47.708297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:47.708297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:47.708297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:47.708297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:47.708297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:47.708297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:47.708297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:47.708297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:47.708297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:40:47.708297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:47.708297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:47.708297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:47.708297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:47.709215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:47.709215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:47.709215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:47.709215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:47.709215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:47.709215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:47.709215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:47.709215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:47.709215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:47.709215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:47.709215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:40:47.709215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:40:47.709881 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 127/248] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:40:47.719685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190813cd-4202-4a4e-b02c-4225cb98225c None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:47.729030 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:47.750350 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec42c1-fffb-413b-8a4a-23ab0a42fb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:47.751048 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec42c1-fffb-413b-8a4a-23ab0a42fb5f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:47.751335 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec42c1-fffb-413b-8a4a-23ab0a42fb5f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:47.751633 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec42c1-fffb-413b-8a4a-23ab0a42fb5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:47.753376 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec42c1-fffb-413b-8a4a-23ab0a42fb5f None admin] RBAC: Adding query filter params (role.id=ebb0d7a0f606450a8dbfee510b7b0b80, scope.project.id=212e33252b164b33ae38cf0925858a09, user.id=9e130e15017b49498c39de87c26d58e4) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:47.753925 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec42c1-fffb-413b-8a4a-23ab0a42fb5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:47.756265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec42c1-fffb-413b-8a4a-23ab0a42fb5f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:47.764913 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 122/249] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:40:47 2022] GET /identity/v3/role_assignments?user.id=9e130e15017b49498c39de87c26d58e4&scope.project.id=212e33252b164b33ae38cf0925858a09&role.id=ebb0d7a0f606450a8dbfee510b7b0b80 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:47.911956 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50536b29-bc15-4f9f-831c-56a3fd86485b None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:47.914099 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50536b29-bc15-4f9f-831c-56a3fd86485b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:47.914345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50536b29-bc15-4f9f-831c-56a3fd86485b None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:47.916749 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 123/250] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:47.927605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145585d-b798-4b30-99f4-5e7e57fa13e5 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:47.928072 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145585d-b798-4b30-99f4-5e7e57fa13e5 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:47.928573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145585d-b798-4b30-99f4-5e7e57fa13e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:47.963298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b145585d-b798-4b30-99f4-5e7e57fa13e5 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:47.964629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190813cd-4202-4a4e-b02c-4225cb98225c None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:47.965439 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b145585d-b798-4b30-99f4-5e7e57fa13e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:48.036294 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aaf7a8-95e0-4f6a-b258-0bbb1e40e1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:48.037948 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aaf7a8-95e0-4f6a-b258-0bbb1e40e1f2 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:48.038819 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aaf7a8-95e0-4f6a-b258-0bbb1e40e1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:48.039394 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aaf7a8-95e0-4f6a-b258-0bbb1e40e1f2 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:48.042934 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aaf7a8-95e0-4f6a-b258-0bbb1e40e1f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:48.043700 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aaf7a8-95e0-4f6a-b258-0bbb1e40e1f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:48.046451 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aaf7a8-95e0-4f6a-b258-0bbb1e40e1f2 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:48.062977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 128/251] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:40:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:48.075775 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aaf7a8-95e0-4f6a-b258-0bbb1e40e1f2 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:48.158811 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 124/252] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:48.235692 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b145585d-b798-4b30-99f4-5e7e57fa13e5 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:48.242564 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:48.312651 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aaf7a8-95e0-4f6a-b258-0bbb1e40e1f2 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:48.402242 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:48.452349 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293dd9c-460d-4a33-b1a6-e29ae387f7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:48.453778 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293dd9c-460d-4a33-b1a6-e29ae387f7a7 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:48.454259 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293dd9c-460d-4a33-b1a6-e29ae387f7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:48.454731 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293dd9c-460d-4a33-b1a6-e29ae387f7a7 None admin] PATH_INFO: `/v3/users/9b3baedf44cb47318665288b67ae20d0` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:48.463449 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0293dd9c-460d-4a33-b1a6-e29ae387f7a7 None admin] RBAC: Authorizing `identity:get_user(user_id=9b3baedf44cb47318665288b67ae20d0)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:48.466213 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0293dd9c-460d-4a33-b1a6-e29ae387f7a7 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:48.469804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 129/253] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:48 2022] GET /identity/v3/users/9b3baedf44cb47318665288b67ae20d0 => generated 297 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:48.479609 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293dd9c-460d-4a33-b1a6-e29ae387f7a7 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:48.590708 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b145585d-b798-4b30-99f4-5e7e57fa13e5 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:48.684648 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37dbb6-8757-47c4-b1e8-c43fcf69ac06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:48.685986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37dbb6-8757-47c4-b1e8-c43fcf69ac06 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:48.686533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37dbb6-8757-47c4-b1e8-c43fcf69ac06 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:48.687022 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37dbb6-8757-47c4-b1e8-c43fcf69ac06 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:48.714468 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37dbb6-8757-47c4-b1e8-c43fcf69ac06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:48.723505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-7c37dbb6-8757-47c4-b1e8-c43fcf69ac06 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:40:48.723505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:48.723505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:48.723505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:48.723505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:48.723505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:48.723505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:48.723505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:48.723505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:48.723505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:48.723505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:48.723505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:48.723505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:48.723505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:48.723505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:48.723505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:48.724451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:48.724451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:48.724451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:48.724451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:48.724451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:48.724451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:48.724451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:48.724451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:48.724451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:48.724451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:48.724451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:48.724451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:40:48.724451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:48.724451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:48.724451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:48.724451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:48.725241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:48.725241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:48.725241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:48.725241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:48.725241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:48.725241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:48.725241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:48.725241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:48.725241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:48.725241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:48.725241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:40:48.725241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:40:48.727858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 125/254] 10.222.0.78 () {56 vars in 1114 bytes} [Wed May 25 14:40:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 493 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:48.747595 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37dbb6-8757-47c4-b1e8-c43fcf69ac06 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:48.843456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293dd9c-460d-4a33-b1a6-e29ae387f7a7 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:48.944020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e91cd-0050-4711-88e8-2ea086df8921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:48.945289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e91cd-0050-4711-88e8-2ea086df8921 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:48.945799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e91cd-0050-4711-88e8-2ea086df8921 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:48.946312 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e91cd-0050-4711-88e8-2ea086df8921 None admin] PATH_INFO: `/v3/projects/212e33252b164b33ae38cf0925858a09` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:48.947926 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005e91cd-0050-4711-88e8-2ea086df8921 None admin] RBAC: Authorizing `identity:get_project(project_id=212e33252b164b33ae38cf0925858a09)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:48.949654 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005e91cd-0050-4711-88e8-2ea086df8921 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:48.958046 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 130/255] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:48 2022] GET /identity/v3/projects/212e33252b164b33ae38cf0925858a09 => generated 302 bytes in 473 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:48.960604 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:48.965675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e91cd-0050-4711-88e8-2ea086df8921 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:48.969729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:49.006505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37dbb6-8757-47c4-b1e8-c43fcf69ac06 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:49.120893 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2c9cf-ec3c-4421-b2c7-1bd1788a14cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:49.122087 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2c9cf-ec3c-4421-b2c7-1bd1788a14cd None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:49.122607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2c9cf-ec3c-4421-b2c7-1bd1788a14cd None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:49.123111 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2c9cf-ec3c-4421-b2c7-1bd1788a14cd None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:49.124612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2c9cf-ec3c-4421-b2c7-1bd1788a14cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:49.125161 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2c9cf-ec3c-4421-b2c7-1bd1788a14cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:49.128074 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2c9cf-ec3c-4421-b2c7-1bd1788a14cd None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:49.161849 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 126/256] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:40:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:49.170800 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2c9cf-ec3c-4421-b2c7-1bd1788a14cd None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:49.212413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e91cd-0050-4711-88e8-2ea086df8921 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:49.296140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6f6cf-04e7-4295-a47e-4c295cb06757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:49.297499 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6f6cf-04e7-4295-a47e-4c295cb06757 None admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:49.298109 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6f6cf-04e7-4295-a47e-4c295cb06757 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:49.298615 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6f6cf-04e7-4295-a47e-4c295cb06757 None admin] PATH_INFO: `/v3/projects/212e33252b164b33ae38cf0925858a09/users/9b3baedf44cb47318665288b67ae20d0/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:49.301165 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc6f6cf-04e7-4295-a47e-4c295cb06757 None admin] RBAC: Authorizing `identity:create_grant(project_id=212e33252b164b33ae38cf0925858a09, user_id=9b3baedf44cb47318665288b67ae20d0, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:49.303065 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc6f6cf-04e7-4295-a47e-4c295cb06757 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:49.339953 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 131/257] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:40:48 2022] PUT /identity/v3/projects/212e33252b164b33ae38cf0925858a09/users/9b3baedf44cb47318665288b67ae20d0/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:49.377119 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3adc5-67c2-499c-9b9f-fcfd0301bf87 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:49.377861 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3adc5-67c2-499c-9b9f-fcfd0301bf87 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:49.378670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3adc5-67c2-499c-9b9f-fcfd0301bf87 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:49.380532 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 132/258] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:40:49.387024 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce60f5-909a-4fd3-9657-c212e716202e None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:49.387648 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce60f5-909a-4fd3-9657-c212e716202e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:49.388161 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce60f5-909a-4fd3-9657-c212e716202e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:49.410940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-91ce60f5-909a-4fd3-9657-c212e716202e None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:49.412836 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-91ce60f5-909a-4fd3-9657-c212e716202e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:49.488053 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2c9cf-ec3c-4421-b2c7-1bd1788a14cd None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:49.551248 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:49.573188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aadd8cf-ae51-4783-8eda-01643ec7406f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:49.579354 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadd8cf-ae51-4783-8eda-01643ec7406f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:49.580618 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadd8cf-ae51-4783-8eda-01643ec7406f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:49.581127 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadd8cf-ae51-4783-8eda-01643ec7406f None admin] PATH_INFO: `/v3/users/69c9c03a92b84438b307c25afc4d0f61` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:49.583098 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aadd8cf-ae51-4783-8eda-01643ec7406f None admin] RBAC: Authorizing `identity:get_user(user_id=69c9c03a92b84438b307c25afc4d0f61)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:49.584983 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aadd8cf-ae51-4783-8eda-01643ec7406f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:49.596214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 127/259] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:49 2022] GET /identity/v3/users/69c9c03a92b84438b307c25afc4d0f61 => generated 283 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:49.604591 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aadd8cf-ae51-4783-8eda-01643ec7406f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:49.608483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 133/260] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:40:49.641909 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce60f5-909a-4fd3-9657-c212e716202e None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:49.646447 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:49.804499 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aadd8cf-ae51-4783-8eda-01643ec7406f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:49.849423 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a971cb-b50b-4761-928b-bcd127468349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:49.850608 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a971cb-b50b-4761-928b-bcd127468349 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:49.851169 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a971cb-b50b-4761-928b-bcd127468349 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:49.851682 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a971cb-b50b-4761-928b-bcd127468349 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:49.944329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a971cb-b50b-4761-928b-bcd127468349 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:49.946256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a971cb-b50b-4761-928b-bcd127468349 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:49.968607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:49.990561 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce60f5-909a-4fd3-9657-c212e716202e None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:50.070277 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 128/261] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:40:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 14:40:50.083440 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a971cb-b50b-4761-928b-bcd127468349 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:50.155269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431c564-56eb-44ba-962b-35b81f37fd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:50.156142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431c564-56eb-44ba-962b-35b81f37fd78 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:50.156600 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431c564-56eb-44ba-962b-35b81f37fd78 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:50.156754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431c564-56eb-44ba-962b-35b81f37fd78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:50.218809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c431c564-56eb-44ba-962b-35b81f37fd78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:50.227280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-c431c564-56eb-44ba-962b-35b81f37fd78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:40:50.227280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:50.227280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:50.227280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:50.227280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:50.227280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:50.227280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:50.227280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:50.227280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:50.227280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:50.227280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:50.227280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:50.227280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:50.227280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:50.227280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:50.227280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:50.228976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:50.228976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:50.228976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:50.228976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:50.228976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:50.228976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:50.228976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:50.228976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:50.228976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:50.228976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:50.228976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:50.228976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:40:50.228976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:50.228976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:50.228976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:50.228976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:50.230709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:50.230709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:50.230709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:50.230709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:50.230709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:50.230709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:50.230709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:50.230709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:50.230709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:50.230709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:50.230709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:40:50.230709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:40:50.230709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 134/262] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:40:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 588 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:50.243425 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431c564-56eb-44ba-962b-35b81f37fd78 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:50.297863 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a971cb-b50b-4761-928b-bcd127468349 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:50.370568 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:50.378872 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a755a-d6a5-4437-9bea-0de54d573f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:50.381754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a755a-d6a5-4437-9bea-0de54d573f54 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:50.382635 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a755a-d6a5-4437-9bea-0de54d573f54 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:50.383448 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a755a-d6a5-4437-9bea-0de54d573f54 None admin] PATH_INFO: `/v3/projects/212e33252b164b33ae38cf0925858a09` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:50.386299 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a755a-d6a5-4437-9bea-0de54d573f54 None admin] RBAC: Authorizing `identity:get_project(project_id=212e33252b164b33ae38cf0925858a09)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:50.390211 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a755a-d6a5-4437-9bea-0de54d573f54 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:50.394685 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 129/263] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:50 2022] GET /identity/v3/projects/212e33252b164b33ae38cf0925858a09 => generated 302 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:50.407738 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a755a-d6a5-4437-9bea-0de54d573f54 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:50.444260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:50.637939 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431c564-56eb-44ba-962b-35b81f37fd78 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:50.641554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a755a-d6a5-4437-9bea-0de54d573f54 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:50.714867 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc9e78-dd5b-464e-9c8b-27b6eb45059b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:50.715582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc9e78-dd5b-464e-9c8b-27b6eb45059b None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:50.715868 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc9e78-dd5b-464e-9c8b-27b6eb45059b None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:50.716146 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc9e78-dd5b-464e-9c8b-27b6eb45059b None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:50.716939 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc9e78-dd5b-464e-9c8b-27b6eb45059b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:50.717253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc9e78-dd5b-464e-9c8b-27b6eb45059b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:50.718735 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc9e78-dd5b-464e-9c8b-27b6eb45059b None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:50.719571 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b8258-79da-4c46-8764-2cd5c2be2c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:50.721242 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b8258-79da-4c46-8764-2cd5c2be2c5c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:50.721979 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b8258-79da-4c46-8764-2cd5c2be2c5c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:50.722492 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b8258-79da-4c46-8764-2cd5c2be2c5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:50.723580 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b8258-79da-4c46-8764-2cd5c2be2c5c None admin] RBAC: Adding query filter params (role.id=d451467ccbbf4149abbfc9b0f76b9339, scope.project.id=212e33252b164b33ae38cf0925858a09, user.id=69c9c03a92b84438b307c25afc4d0f61) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:50.724408 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b8258-79da-4c46-8764-2cd5c2be2c5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:50.727619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b8258-79da-4c46-8764-2cd5c2be2c5c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:50.729259 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 135/264] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 486 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:50.746781 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc9e78-dd5b-464e-9c8b-27b6eb45059b None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:50.749935 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 130/265] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:40:50 2022] GET /identity/v3/role_assignments?user.id=69c9c03a92b84438b307c25afc4d0f61&scope.project.id=212e33252b164b33ae38cf0925858a09&role.id=d451467ccbbf4149abbfc9b0f76b9339 => generated 608 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:50.903706 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc9e78-dd5b-464e-9c8b-27b6eb45059b None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:50.954712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:50.974677 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39fd78-3323-436e-9f28-bffbb41b0518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:50.975688 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39fd78-3323-436e-9f28-bffbb41b0518 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:50.975970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39fd78-3323-436e-9f28-bffbb41b0518 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:50.976243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39fd78-3323-436e-9f28-bffbb41b0518 None admin] PATH_INFO: `/v3/users/69c9c03a92b84438b307c25afc4d0f61` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:50.978922 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc39fd78-3323-436e-9f28-bffbb41b0518 None admin] RBAC: Authorizing `identity:get_user(user_id=69c9c03a92b84438b307c25afc4d0f61)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:50.980427 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc39fd78-3323-436e-9f28-bffbb41b0518 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:50.982645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 136/266] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:50 2022] GET /identity/v3/users/69c9c03a92b84438b307c25afc4d0f61 => generated 283 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:50.992090 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b8258-79da-4c46-8764-2cd5c2be2c5c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:51.156381 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:51.281076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b8258-79da-4c46-8764-2cd5c2be2c5c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:51.334068 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7363b6-5eab-4be4-b43b-f8c0717dc63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:51.334776 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7363b6-5eab-4be4-b43b-f8c0717dc63b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:51.335069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7363b6-5eab-4be4-b43b-f8c0717dc63b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:51.335391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7363b6-5eab-4be4-b43b-f8c0717dc63b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:51.442392 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7363b6-5eab-4be4-b43b-f8c0717dc63b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:51.443840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7363b6-5eab-4be4-b43b-f8c0717dc63b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:51.513435 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:51.591650 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 131/267] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:40:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:40:51.597946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39fd78-3323-436e-9f28-bffbb41b0518 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:51.747485 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39fd78-3323-436e-9f28-bffbb41b0518 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:51.791623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ccc6b-d479-4696-9ac9-3a2df20553d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:51.792534 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ccc6b-d479-4696-9ac9-3a2df20553d8 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:51.792836 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ccc6b-d479-4696-9ac9-3a2df20553d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:51.793076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ccc6b-d479-4696-9ac9-3a2df20553d8 None admin] PATH_INFO: `/v3/projects/212e33252b164b33ae38cf0925858a09` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:51.794475 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ccc6b-d479-4696-9ac9-3a2df20553d8 None admin] RBAC: Authorizing `identity:get_project(project_id=212e33252b164b33ae38cf0925858a09)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:51.796000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ccc6b-d479-4696-9ac9-3a2df20553d8 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:51.798277 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 137/268] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:51 2022] GET /identity/v3/projects/212e33252b164b33ae38cf0925858a09 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:51.804787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7363b6-5eab-4be4-b43b-f8c0717dc63b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:52.016324 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7363b6-5eab-4be4-b43b-f8c0717dc63b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:52.077827 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33569b3f-7467-4c4d-a0f0-2ac4816fdb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:52.078590 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33569b3f-7467-4c4d-a0f0-2ac4816fdb64 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:52.078881 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33569b3f-7467-4c4d-a0f0-2ac4816fdb64 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:52.079161 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33569b3f-7467-4c4d-a0f0-2ac4816fdb64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:52.080061 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33569b3f-7467-4c4d-a0f0-2ac4816fdb64 None admin] RBAC: Adding query filter params (role.id=e566b37cd92f467198b5dc40f183effc, scope.project.id=212e33252b164b33ae38cf0925858a09, user.id=69c9c03a92b84438b307c25afc4d0f61) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:52.080379 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33569b3f-7467-4c4d-a0f0-2ac4816fdb64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:52.081909 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33569b3f-7467-4c4d-a0f0-2ac4816fdb64 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:52.089430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 132/269] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:40:51 2022] GET /identity/v3/role_assignments?user.id=69c9c03a92b84438b307c25afc4d0f61&scope.project.id=212e33252b164b33ae38cf0925858a09&role.id=e566b37cd92f467198b5dc40f183effc => generated 608 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:52.263527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a43d0-71ed-4625-87f8-94ea5d5776b6 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:52.263828 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a43d0-71ed-4625-87f8-94ea5d5776b6 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:52.264227 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a43d0-71ed-4625-87f8-94ea5d5776b6 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:52.265587 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 138/270] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:52.284946 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f0397-afa3-4300-ab23-efa7684d9ff5 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:52.285174 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f0397-afa3-4300-ab23-efa7684d9ff5 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:52.285483 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f0397-afa3-4300-ab23-efa7684d9ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:52.309571 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-c40f0397-afa3-4300-ab23-efa7684d9ff5 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:52.310858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c40f0397-afa3-4300-ab23-efa7684d9ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:52.322921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:52.351855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a369bf1-d64d-4216-942e-bd441a429796 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:52.352152 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a369bf1-d64d-4216-942e-bd441a429796 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:52.352468 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a369bf1-d64d-4216-942e-bd441a429796 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:52.354122 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 139/271] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:40:52.361412 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b931a1b-9719-4cb7-a6a5-289037ee5b84 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:52.361685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b931a1b-9719-4cb7-a6a5-289037ee5b84 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:52.361993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b931a1b-9719-4cb7-a6a5-289037ee5b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:52.380161 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-2b931a1b-9719-4cb7-a6a5-289037ee5b84 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:52.381475 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-2b931a1b-9719-4cb7-a6a5-289037ee5b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:52.400237 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 133/272] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:52.407941 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead4df6-be72-4c5a-8055-433da79ac7f0 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:52.408662 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead4df6-be72-4c5a-8055-433da79ac7f0 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:52.408956 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead4df6-be72-4c5a-8055-433da79ac7f0 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:52.410458 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 134/273] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:52.418382 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b580c-93ae-4070-ba0d-82f0809a121f None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:52.418771 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b580c-93ae-4070-ba0d-82f0809a121f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:52.419065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b580c-93ae-4070-ba0d-82f0809a121f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:52.438574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-fb6b580c-93ae-4070-ba0d-82f0809a121f None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:52.440191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-fb6b580c-93ae-4070-ba0d-82f0809a121f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:52.447706 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:52.493679 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 140/274] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:52.494896 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b931a1b-9719-4cb7-a6a5-289037ee5b84 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:52.550602 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 135/275] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:52.552364 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b580c-93ae-4070-ba0d-82f0809a121f None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:52.667523 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b931a1b-9719-4cb7-a6a5-289037ee5b84 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:52.670578 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:52.714139 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b580c-93ae-4070-ba0d-82f0809a121f None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:52.732030 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b7b62-5670-4444-b39c-112c8bb827c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:52.733201 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b7b62-5670-4444-b39c-112c8bb827c8 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:52.733661 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b7b62-5670-4444-b39c-112c8bb827c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:52.734175 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b7b62-5670-4444-b39c-112c8bb827c8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:52.749574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86b7b62-5670-4444-b39c-112c8bb827c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:52.765028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-b86b7b62-5670-4444-b39c-112c8bb827c8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:40:52.765028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:52.765028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:52.765028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:52.765028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:52.765028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:52.765028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:52.765028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:52.765028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:52.765028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:52.765028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:52.765028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:52.765028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:52.765028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:52.765028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:52.765028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:52.766337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:52.766337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:52.766337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:52.766337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:52.766337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:52.766337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:52.766337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:52.766337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:52.766337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:52.766337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:52.766337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:52.766337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:40:52.766337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:52.766337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:52.766337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:52.766337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:52.767205 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:52.767205 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:52.767205 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:52.767205 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:52.767205 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:52.767205 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:52.767205 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:52.767205 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:52.767205 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:52.767205 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:52.767205 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:40:52.767205 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:40:52.768377 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 141/276] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:52.777395 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b7b62-5670-4444-b39c-112c8bb827c8 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:52.784533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b55c14-dbce-44e2-b1e6-913908df072b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:52.786067 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b55c14-dbce-44e2-b1e6-913908df072b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:52.786578 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b55c14-dbce-44e2-b1e6-913908df072b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:52.787080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b55c14-dbce-44e2-b1e6-913908df072b None admin] PATH_INFO: `/v3/roles/member` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:52.820534 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b55c14-dbce-44e2-b1e6-913908df072b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:52.842949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-88b55c14-dbce-44e2-b1e6-913908df072b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:40:52.842949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:52.842949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:52.842949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:52.842949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:52.842949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:52.842949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:52.842949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:52.842949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:52.842949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:52.842949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:52.842949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:52.842949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:52.842949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:52.842949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:52.842949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:52.843934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:52.843934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:52.843934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:52.843934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:52.843934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:52.843934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:52.843934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:52.843934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:52.843934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:52.843934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:52.843934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:52.843934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:40:52.843934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:52.843934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:52.843934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:52.843934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:52.844855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:52.844855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:52.844855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:52.844855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:52.844855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:52.844855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:52.844855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:52.844855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:52.844855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:52.844855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:52.844855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:40:52.844855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:40:52.845459 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:52.845532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 136/277] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:52.851330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b55c14-dbce-44e2-b1e6-913908df072b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:52.956458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b7b62-5670-4444-b39c-112c8bb827c8 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:53.012074 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71e0d3-9b76-4f12-845a-2d90723e667d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:53.014173 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71e0d3-9b76-4f12-845a-2d90723e667d None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:53.014680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71e0d3-9b76-4f12-845a-2d90723e667d None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:53.015633 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71e0d3-9b76-4f12-845a-2d90723e667d None admin] PATH_INFO: `/v3/domains/default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:53.018934 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71e0d3-9b76-4f12-845a-2d90723e667d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:53.025438 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71e0d3-9b76-4f12-845a-2d90723e667d None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:53.028994 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 142/278] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:40:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:53.030486 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71e0d3-9b76-4f12-845a-2d90723e667d None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:53.049327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b55c14-dbce-44e2-b1e6-913908df072b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:53.099453 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01af3da-8d03-4c37-8a0e-7bb21b86a20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:53.100258 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01af3da-8d03-4c37-8a0e-7bb21b86a20f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:53.100676 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01af3da-8d03-4c37-8a0e-7bb21b86a20f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:53.101192 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01af3da-8d03-4c37-8a0e-7bb21b86a20f None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:53.102184 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01af3da-8d03-4c37-8a0e-7bb21b86a20f None admin] RBAC: Adding query filter params (name=reader) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:53.102639 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01af3da-8d03-4c37-8a0e-7bb21b86a20f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:53.104261 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01af3da-8d03-4c37-8a0e-7bb21b86a20f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:53.106522 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:53.124173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 137/279] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:40:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:53.125232 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01af3da-8d03-4c37-8a0e-7bb21b86a20f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:53.242209 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71e0d3-9b76-4f12-845a-2d90723e667d None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:53.261339 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:53.280575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01af3da-8d03-4c37-8a0e-7bb21b86a20f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:53.313426 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e3fca-1a3c-4c3d-a9f4-2a6e9de9d417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:53.314691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e3fca-1a3c-4c3d-a9f4-2a6e9de9d417 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:53.315209 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e3fca-1a3c-4c3d-a9f4-2a6e9de9d417 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:53.315716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e3fca-1a3c-4c3d-a9f4-2a6e9de9d417 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:53.317212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e3fca-1a3c-4c3d-a9f4-2a6e9de9d417 None admin] RBAC: Adding query filter params (name=member) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:53.317791 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e3fca-1a3c-4c3d-a9f4-2a6e9de9d417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:53.320480 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e3fca-1a3c-4c3d-a9f4-2a6e9de9d417 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:53.329283 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e143aa-9fd3-43f1-a24c-bf811dbb45c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:53.330835 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e143aa-9fd3-43f1-a24c-bf811dbb45c5 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:53.331325 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e143aa-9fd3-43f1-a24c-bf811dbb45c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:53.331801 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e143aa-9fd3-43f1-a24c-bf811dbb45c5 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:53.332841 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e143aa-9fd3-43f1-a24c-bf811dbb45c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:53.334964 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e143aa-9fd3-43f1-a24c-bf811dbb45c5 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:53.338766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 143/280] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:40:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:53.348382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e3fca-1a3c-4c3d-a9f4-2a6e9de9d417 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:53.378441 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 138/281] 10.222.0.78 () {60 vars in 1132 bytes} [Wed May 25 14:40:53 2022] POST /identity/v3/users => generated 293 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:53.378441 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e143aa-9fd3-43f1-a24c-bf811dbb45c5 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:53.515757 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:53.595958 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e3fca-1a3c-4c3d-a9f4-2a6e9de9d417 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:53.632868 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e143aa-9fd3-43f1-a24c-bf811dbb45c5 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:53.674115 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6996fba-91b1-47f4-b571-466842979d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:53.675534 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6996fba-91b1-47f4-b571-466842979d1f None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:53.675946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6996fba-91b1-47f4-b571-466842979d1f None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:53.676398 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6996fba-91b1-47f4-b571-466842979d1f None admin] PATH_INFO: `/v3/users/9e130e15017b49498c39de87c26d58e4` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:53.678047 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6996fba-91b1-47f4-b571-466842979d1f None admin] RBAC: Authorizing `identity:get_user(user_id=9e130e15017b49498c39de87c26d58e4)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:53.680477 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6996fba-91b1-47f4-b571-466842979d1f None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:53.683420 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 144/282] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:53 2022] GET /identity/v3/users/9e130e15017b49498c39de87c26d58e4 => generated 297 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:53.685468 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9610a5-c6f9-415b-9e4e-5d054be616b1 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:53.685878 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9610a5-c6f9-415b-9e4e-5d054be616b1 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:53.686304 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9610a5-c6f9-415b-9e4e-5d054be616b1 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:53.688501 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 145/283] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:53.690434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9610a5-c6f9-415b-9e4e-5d054be616b1 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:53.700645 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c3b52-c689-461b-80c9-b00f3b674f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:53.702127 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c3b52-c689-461b-80c9-b00f3b674f8d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:53.702601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c3b52-c689-461b-80c9-b00f3b674f8d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:53.703054 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c3b52-c689-461b-80c9-b00f3b674f8d None admin] PATH_INFO: `/v3/users/68ac7291f5b24ecfbf7b31b93607d1a9` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:53.732941 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c3b52-c689-461b-80c9-b00f3b674f8d None admin] RBAC: Authorizing `identity:get_user(user_id=68ac7291f5b24ecfbf7b31b93607d1a9)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:53.736529 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c3b52-c689-461b-80c9-b00f3b674f8d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:53.740283 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 139/284] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:53 2022] GET /identity/v3/users/68ac7291f5b24ecfbf7b31b93607d1a9 => generated 293 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:53.748615 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f8b46-cf47-4d40-bebb-3ed0b579289c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:53.749135 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f8b46-cf47-4d40-bebb-3ed0b579289c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:53.749652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f8b46-cf47-4d40-bebb-3ed0b579289c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:53.771648 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:53.781940 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-d34f8b46-cf47-4d40-bebb-3ed0b579289c None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:53.784737 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d34f8b46-cf47-4d40-bebb-3ed0b579289c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:53.894916 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9610a5-c6f9-415b-9e4e-5d054be616b1 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:53.921083 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 140/285] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:53.921949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f8b46-cf47-4d40-bebb-3ed0b579289c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:53.956192 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccead9a8-2509-4d9b-9004-583ff4e92737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:53.958501 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccead9a8-2509-4d9b-9004-583ff4e92737 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:53.959451 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccead9a8-2509-4d9b-9004-583ff4e92737 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:53.959925 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccead9a8-2509-4d9b-9004-583ff4e92737 None admin] PATH_INFO: `/v3/projects/212e33252b164b33ae38cf0925858a09` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:53.962306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccead9a8-2509-4d9b-9004-583ff4e92737 None admin] RBAC: Authorizing `identity:get_project(project_id=212e33252b164b33ae38cf0925858a09)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:53.965319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccead9a8-2509-4d9b-9004-583ff4e92737 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:53.967625 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 146/286] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:53 2022] GET /identity/v3/projects/212e33252b164b33ae38cf0925858a09 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:53.978264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccead9a8-2509-4d9b-9004-583ff4e92737 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:54.007776 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:54.142235 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f8b46-cf47-4d40-bebb-3ed0b579289c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:54.211314 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e4a02-5ff3-4a76-8279-68c43f2c5e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:54.212875 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e4a02-5ff3-4a76-8279-68c43f2c5e71 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:54.213676 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e4a02-5ff3-4a76-8279-68c43f2c5e71 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:54.214507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e4a02-5ff3-4a76-8279-68c43f2c5e71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:54.215600 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccead9a8-2509-4d9b-9004-583ff4e92737 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:54.216650 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e4a02-5ff3-4a76-8279-68c43f2c5e71 None admin] RBAC: Adding query filter params (role.id=a3da3c62738d4790b9dac512b1568b52, user.id=68ac7291f5b24ecfbf7b31b93607d1a9, scope.system=all) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:54.217556 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e4a02-5ff3-4a76-8279-68c43f2c5e71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:54.221112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e4a02-5ff3-4a76-8279-68c43f2c5e71 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:54.231375 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 141/287] 10.222.0.78 () {56 vars in 1308 bytes} [Wed May 25 14:40:53 2022] GET /identity/v3/role_assignments?user.id=68ac7291f5b24ecfbf7b31b93607d1a9&scope.system=all&role.id=a3da3c62738d4790b9dac512b1568b52 => generated 226 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:54.231893 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:54.233548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e4a02-5ff3-4a76-8279-68c43f2c5e71 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:54.294421 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fea55-c5ba-4b15-abe1-ad9721ed4323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:54.296476 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fea55-c5ba-4b15-abe1-ad9721ed4323 None admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:54.297291 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fea55-c5ba-4b15-abe1-ad9721ed4323 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:54.298003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fea55-c5ba-4b15-abe1-ad9721ed4323 None admin] PATH_INFO: `/v3/projects/212e33252b164b33ae38cf0925858a09/users/9e130e15017b49498c39de87c26d58e4/roles/ebb0d7a0f606450a8dbfee510b7b0b80` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:54.302937 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913fea55-c5ba-4b15-abe1-ad9721ed4323 None admin] RBAC: Authorizing `identity:create_grant(project_id=212e33252b164b33ae38cf0925858a09, user_id=9e130e15017b49498c39de87c26d58e4, role_id=ebb0d7a0f606450a8dbfee510b7b0b80)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:54.308154 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913fea55-c5ba-4b15-abe1-ad9721ed4323 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:54.326384 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 147/288] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:40:53 2022] PUT /identity/v3/projects/212e33252b164b33ae38cf0925858a09/users/9e130e15017b49498c39de87c26d58e4/roles/ebb0d7a0f606450a8dbfee510b7b0b80 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:54.476142 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e4a02-5ff3-4a76-8279-68c43f2c5e71 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:54.481227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:54.562439 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7180675-53d0-4a35-8acf-36b8ca8412fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:54.563781 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7180675-53d0-4a35-8acf-36b8ca8412fa None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:54.564810 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7180675-53d0-4a35-8acf-36b8ca8412fa None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:54.565272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7180675-53d0-4a35-8acf-36b8ca8412fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:54.589576 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7180675-53d0-4a35-8acf-36b8ca8412fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:54.604686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-a7180675-53d0-4a35-8acf-36b8ca8412fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:40:54.604686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:54.604686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:54.604686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:54.604686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:54.604686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:54.604686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:54.604686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:54.604686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:54.604686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:54.604686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:54.604686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:54.604686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:54.604686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:54.604686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:54.604686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:54.605723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:54.605723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:54.605723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:54.605723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:54.605723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:54.605723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:54.605723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:54.605723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:54.605723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:54.605723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:54.605723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:54.605723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:40:54.605723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:54.605723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:54.605723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:54.605723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:54.606606 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:54.606606 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:54.606606 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:54.606606 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:54.606606 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:54.606606 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:54.606606 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:54.606606 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:54.606606 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:54.606606 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:54.606606 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:40:54.606606 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:40:54.607289 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 142/289] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:54.619226 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fea55-c5ba-4b15-abe1-ad9721ed4323 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:54.770579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fea55-c5ba-4b15-abe1-ad9721ed4323 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:54.843456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232696c-0538-4427-b637-2303d81ed337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:54.844659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232696c-0538-4427-b637-2303d81ed337 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:54.845137 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232696c-0538-4427-b637-2303d81ed337 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:54.846218 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232696c-0538-4427-b637-2303d81ed337 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:54.866690 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f232696c-0538-4427-b637-2303d81ed337 None admin] RBAC: Adding query filter params (name=member) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:54.867534 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f232696c-0538-4427-b637-2303d81ed337 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:54.870442 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f232696c-0538-4427-b637-2303d81ed337 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:54.921736 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 148/290] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:40:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:54.933008 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7180675-53d0-4a35-8acf-36b8ca8412fa None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:55.304110 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7180675-53d0-4a35-8acf-36b8ca8412fa None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:55.399287 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5296eff-f9ea-422e-a103-edcdf78593f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:55.400826 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5296eff-f9ea-422e-a103-edcdf78593f2 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:55.401319 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5296eff-f9ea-422e-a103-edcdf78593f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:55.401802 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5296eff-f9ea-422e-a103-edcdf78593f2 None admin] PATH_INFO: `/v3/users/9b3baedf44cb47318665288b67ae20d0` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:55.404110 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5296eff-f9ea-422e-a103-edcdf78593f2 None admin] RBAC: Authorizing `identity:get_user(user_id=9b3baedf44cb47318665288b67ae20d0)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:55.406864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5296eff-f9ea-422e-a103-edcdf78593f2 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:55.410778 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 143/291] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:54 2022] GET /identity/v3/users/9b3baedf44cb47318665288b67ae20d0 => generated 297 bytes in 479 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:55.419746 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232696c-0538-4427-b637-2303d81ed337 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:55.500063 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:55.686402 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232696c-0538-4427-b637-2303d81ed337 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:55.775087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ae740-784d-4a2a-8de2-74adf800a7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:55.776152 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ae740-784d-4a2a-8de2-74adf800a7ec None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:55.776631 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ae740-784d-4a2a-8de2-74adf800a7ec None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:55.777106 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ae740-784d-4a2a-8de2-74adf800a7ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:55.939617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ae740-784d-4a2a-8de2-74adf800a7ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:55.942300 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ae740-784d-4a2a-8de2-74adf800a7ec None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:55.993164 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:56.181958 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 149/292] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:40:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 763 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 14:40:56.196071 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5296eff-f9ea-422e-a103-edcdf78593f2 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:56.268111 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:56.375436 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5296eff-f9ea-422e-a103-edcdf78593f2 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:56.420124 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57047200-9b09-494b-8887-5e248e88eeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:56.421019 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57047200-9b09-494b-8887-5e248e88eeb3 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:56.421307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57047200-9b09-494b-8887-5e248e88eeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:56.421705 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57047200-9b09-494b-8887-5e248e88eeb3 None admin] PATH_INFO: `/v3/projects/212e33252b164b33ae38cf0925858a09` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:56.426972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57047200-9b09-494b-8887-5e248e88eeb3 None admin] RBAC: Authorizing `identity:get_project(project_id=212e33252b164b33ae38cf0925858a09)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:56.428488 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57047200-9b09-494b-8887-5e248e88eeb3 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:56.430386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 144/293] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:40:56 2022] GET /identity/v3/projects/212e33252b164b33ae38cf0925858a09 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:56.436365 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ae740-784d-4a2a-8de2-74adf800a7ec None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:56.683039 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ae740-784d-4a2a-8de2-74adf800a7ec None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:56.751252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:56.766910 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff503e-378f-4b1a-8813-6c963783404d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:56.768113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff503e-378f-4b1a-8813-6c963783404d None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:56.768605 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff503e-378f-4b1a-8813-6c963783404d None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:56.769085 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff503e-378f-4b1a-8813-6c963783404d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:56.770822 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bff503e-378f-4b1a-8813-6c963783404d None admin] RBAC: Adding query filter params (role.id=a3da3c62738d4790b9dac512b1568b52, scope.project.id=212e33252b164b33ae38cf0925858a09, user.id=9b3baedf44cb47318665288b67ae20d0) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:56.771449 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bff503e-378f-4b1a-8813-6c963783404d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:56.774330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bff503e-378f-4b1a-8813-6c963783404d None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:56.791034 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 150/294] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:40:56 2022] GET /identity/v3/role_assignments?user.id=9b3baedf44cb47318665288b67ae20d0&scope.project.id=212e33252b164b33ae38cf0925858a09&role.id=a3da3c62738d4790b9dac512b1568b52 => generated 608 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:57.206787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25e31c-d130-4a94-b378-99fc29d7373a None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:57.207324 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25e31c-d130-4a94-b378-99fc29d7373a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:57.207766 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25e31c-d130-4a94-b378-99fc29d7373a None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:57.210603 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 145/295] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:57.218270 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb663dee-69f0-42af-a017-aeb6491c8fd9 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:57.218771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb663dee-69f0-42af-a017-aeb6491c8fd9 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:57.219292 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb663dee-69f0-42af-a017-aeb6491c8fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:57.252783 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-fb663dee-69f0-42af-a017-aeb6491c8fd9 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:57.255218 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-fb663dee-69f0-42af-a017-aeb6491c8fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:57.395367 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 151/296] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:57.442316 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb25e31c-d130-4a94-b378-99fc29d7373a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:57.549125 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9847e8-e5de-456c-8bb6-2408ecbf7a96 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:57.549639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9847e8-e5de-456c-8bb6-2408ecbf7a96 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:57.550790 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9847e8-e5de-456c-8bb6-2408ecbf7a96 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:57.554962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 152/297] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:57.563909 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84665eb9-393d-41e2-b8d6-f351c2ba270b None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:57.565845 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84665eb9-393d-41e2-b8d6-f351c2ba270b None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:57.566786 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84665eb9-393d-41e2-b8d6-f351c2ba270b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:57.591907 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-84665eb9-393d-41e2-b8d6-f351c2ba270b None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:57.593670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-84665eb9-393d-41e2-b8d6-f351c2ba270b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:57.665279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:57.672521 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb25e31c-d130-4a94-b378-99fc29d7373a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:57.702133 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 153/298] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:40:57.724895 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84665eb9-393d-41e2-b8d6-f351c2ba270b None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:57.747082 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff496cf4-5a6d-4ca2-8dd7-294b2a1aa858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:57.748811 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff496cf4-5a6d-4ca2-8dd7-294b2a1aa858 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:57.749307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff496cf4-5a6d-4ca2-8dd7-294b2a1aa858 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:57.749790 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff496cf4-5a6d-4ca2-8dd7-294b2a1aa858 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:57.771072 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff496cf4-5a6d-4ca2-8dd7-294b2a1aa858 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:57.787241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-ff496cf4-5a6d-4ca2-8dd7-294b2a1aa858 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:40:57.787241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:57.787241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:57.787241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:57.787241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:57.787241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:57.787241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:57.787241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:57.787241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:57.787241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:57.787241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:57.787241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:57.787241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:57.787241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:57.787241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:57.787241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:57.788536 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:57.788536 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:57.788536 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:57.788536 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:57.788536 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:57.788536 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:57.788536 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:57.788536 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:57.788536 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:57.788536 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:57.788536 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:57.788536 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:40:57.788536 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:57.788536 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:57.788536 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:57.788536 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:57.789329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:57.789329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:57.789329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:57.789329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:57.789329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:57.789329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:57.789329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:57.789329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:57.789329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:57.789329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:57.789329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:40:57.789329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:40:57.790899 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 146/299] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:40:57.800945 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff496cf4-5a6d-4ca2-8dd7-294b2a1aa858 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:57.813419 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:57.890368 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84665eb9-393d-41e2-b8d6-f351c2ba270b None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:57.946702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed583b36-59aa-4508-a65e-d063b5e85862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:57.947945 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed583b36-59aa-4508-a65e-d063b5e85862 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:57.948448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed583b36-59aa-4508-a65e-d063b5e85862 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:57.948947 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed583b36-59aa-4508-a65e-d063b5e85862 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:57.950555 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed583b36-59aa-4508-a65e-d063b5e85862 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:57.953086 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed583b36-59aa-4508-a65e-d063b5e85862 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:57.960716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 154/300] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:40:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:57.963190 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed583b36-59aa-4508-a65e-d063b5e85862 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:58.028737 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff496cf4-5a6d-4ca2-8dd7-294b2a1aa858 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:58.077809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b6a21-7208-46ac-ae7a-c7fcf0eb8a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:58.079228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b6a21-7208-46ac-ae7a-c7fcf0eb8a53 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:58.080818 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b6a21-7208-46ac-ae7a-c7fcf0eb8a53 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:58.081465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b6a21-7208-46ac-ae7a-c7fcf0eb8a53 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:58.082632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b6a21-7208-46ac-ae7a-c7fcf0eb8a53 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:58.083173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b6a21-7208-46ac-ae7a-c7fcf0eb8a53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:58.084829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b6a21-7208-46ac-ae7a-c7fcf0eb8a53 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:58.097185 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 147/301] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:40:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:58.106653 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b6a21-7208-46ac-ae7a-c7fcf0eb8a53 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:58.140674 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed583b36-59aa-4508-a65e-d063b5e85862 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:58.156753 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:58.174515 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:58.195501 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646a25d-69d1-448c-aecd-1c5dc9ff063d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:58.196470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646a25d-69d1-448c-aecd-1c5dc9ff063d None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:58.197203 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646a25d-69d1-448c-aecd-1c5dc9ff063d None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:58.197656 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646a25d-69d1-448c-aecd-1c5dc9ff063d None admin] PATH_INFO: `/v3/groups` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:58.198553 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646a25d-69d1-448c-aecd-1c5dc9ff063d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:58.200510 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646a25d-69d1-448c-aecd-1c5dc9ff063d None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:58.226054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 155/302] 10.222.0.78 () {60 vars in 1134 bytes} [Wed May 25 14:40:57 2022] POST /identity/v3/groups => generated 229 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:58.229138 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec28f39-c8c9-4da6-8cab-d01881c1431e None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:58.229620 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec28f39-c8c9-4da6-8cab-d01881c1431e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:58.230140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec28f39-c8c9-4da6-8cab-d01881c1431e None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:58.232855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 156/303] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:58.243155 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea098c4b-ee0b-4032-a4f5-851be34927d3 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:58.243665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea098c4b-ee0b-4032-a4f5-851be34927d3 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:58.244162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea098c4b-ee0b-4032-a4f5-851be34927d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:58.274633 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-ea098c4b-ee0b-4032-a4f5-851be34927d3 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:58.277083 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-ea098c4b-ee0b-4032-a4f5-851be34927d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:58.350000 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b6a21-7208-46ac-ae7a-c7fcf0eb8a53 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:58.396777 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 157/304] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:40:58.431516 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c824ec-d918-4afc-a203-078a7c1d844a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:58.434370 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c824ec-d918-4afc-a203-078a7c1d844a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:58.435260 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c824ec-d918-4afc-a203-078a7c1d844a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:58.435749 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea098c4b-ee0b-4032-a4f5-851be34927d3 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:58.436217 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c824ec-d918-4afc-a203-078a7c1d844a None admin] PATH_INFO: `/v3/users/601eb237a1074978b1b65807d2045288` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:58.464990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c824ec-d918-4afc-a203-078a7c1d844a None admin] RBAC: Authorizing `identity:get_user(user_id=601eb237a1074978b1b65807d2045288)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:58.468032 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c824ec-d918-4afc-a203-078a7c1d844a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:58.476756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 148/305] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:58 2022] GET /identity/v3/users/601eb237a1074978b1b65807d2045288 => generated 293 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:58.480253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c824ec-d918-4afc-a203-078a7c1d844a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:58.617430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:58.633592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea098c4b-ee0b-4032-a4f5-851be34927d3 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:58.650159 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c824ec-d918-4afc-a203-078a7c1d844a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:58.654437 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:58.694438 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24730784-2c82-4c46-895a-558ede2c96d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:58.696184 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24730784-2c82-4c46-895a-558ede2c96d0 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:58.696603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24730784-2c82-4c46-895a-558ede2c96d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:58.697178 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24730784-2c82-4c46-895a-558ede2c96d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:58.716072 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24730784-2c82-4c46-895a-558ede2c96d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:58.718696 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f889b-10cb-4bf7-896d-395a6c9d21b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:58.719373 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f889b-10cb-4bf7-896d-395a6c9d21b1 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:58.719660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f889b-10cb-4bf7-896d-395a6c9d21b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:58.719940 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f889b-10cb-4bf7-896d-395a6c9d21b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:58.721341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f889b-10cb-4bf7-896d-395a6c9d21b1 None admin] RBAC: Adding query filter params (role.id=ebb0d7a0f606450a8dbfee510b7b0b80, user.id=601eb237a1074978b1b65807d2045288, scope.system=all) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:58.721662 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f889b-10cb-4bf7-896d-395a6c9d21b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:58.723509 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f889b-10cb-4bf7-896d-395a6c9d21b1 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:58.727090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-24730784-2c82-4c46-895a-558ede2c96d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:40:58.727090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:58.727090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:58.727090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:58.727090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:58.727090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:58.727090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:58.727090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:58.727090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:58.727090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:58.727090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:58.727090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:58.727090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:58.727090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:58.727090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:58.727090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:58.729624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:58.729624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:58.729624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:58.729624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:58.729624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:58.729624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:58.729624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:58.729624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:58.729624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:58.729624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:58.729624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:58.729624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:40:58.729624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:58.729624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:58.729624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:58.729624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:58.731559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:58.731559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:58.731559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:58.731559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:58.731559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:58.731559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:58.731559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:58.731559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:58.731559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:58.731559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:58.731559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:40:58.731559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:40:58.731559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 158/306] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:40:58.733289 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 149/307] 10.222.0.78 () {56 vars in 1308 bytes} [Wed May 25 14:40:58 2022] GET /identity/v3/role_assignments?user.id=601eb237a1074978b1b65807d2045288&scope.system=all&role.id=ebb0d7a0f606450a8dbfee510b7b0b80 => generated 226 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:58.736345 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24730784-2c82-4c46-895a-558ede2c96d0 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:58.875786 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24730784-2c82-4c46-895a-558ede2c96d0 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:58.921601 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7173e2-dff2-4d26-a263-6ad9e2a52781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:58.922562 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7173e2-dff2-4d26-a263-6ad9e2a52781 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:58.922950 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7173e2-dff2-4d26-a263-6ad9e2a52781 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:58.923898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7173e2-dff2-4d26-a263-6ad9e2a52781 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:58.924866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7173e2-dff2-4d26-a263-6ad9e2a52781 None admin] RBAC: Adding query filter params (name=member) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:58.925312 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7173e2-dff2-4d26-a263-6ad9e2a52781 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:58.926950 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7173e2-dff2-4d26-a263-6ad9e2a52781 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:58.933020 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc7ca0-a090-418e-a76c-97dc49cec506 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:58.933515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc7ca0-a090-418e-a76c-97dc49cec506 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:58.934030 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc7ca0-a090-418e-a76c-97dc49cec506 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:58.936878 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 150/308] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:40:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:40:58.939811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 159/309] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:40:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:58.946083 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bed94-0e71-4666-98c7-387f683d1d7f None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:58.946412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bed94-0e71-4666-98c7-387f683d1d7f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:58.947833 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bed94-0e71-4666-98c7-387f683d1d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:58.949326 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7173e2-dff2-4d26-a263-6ad9e2a52781 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:58.978218 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-789bed94-0e71-4666-98c7-387f683d1d7f None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:58.979918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-789bed94-0e71-4666-98c7-387f683d1d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:58.998174 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:59.101633 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 151/310] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:40:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:40:59.126334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7173e2-dff2-4d26-a263-6ad9e2a52781 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:59.144870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bed94-0e71-4666-98c7-387f683d1d7f None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:59.186709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1d862-6ed4-4dc3-a58f-65779ac8d0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:59.189519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1d862-6ed4-4dc3-a58f-65779ac8d0b8 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:59.190498 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1d862-6ed4-4dc3-a58f-65779ac8d0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:59.191271 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1d862-6ed4-4dc3-a58f-65779ac8d0b8 None admin] PATH_INFO: `/v3/users/68ac7291f5b24ecfbf7b31b93607d1a9` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:59.193955 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1d862-6ed4-4dc3-a58f-65779ac8d0b8 None admin] RBAC: Authorizing `identity:get_user(user_id=68ac7291f5b24ecfbf7b31b93607d1a9)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:59.197201 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1d862-6ed4-4dc3-a58f-65779ac8d0b8 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:59.200887 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 160/311] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:58 2022] GET /identity/v3/users/68ac7291f5b24ecfbf7b31b93607d1a9 => generated 293 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:59.206938 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1d862-6ed4-4dc3-a58f-65779ac8d0b8 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:59.252406 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:59.286847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bed94-0e71-4666-98c7-387f683d1d7f None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:59.338292 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:59.343216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1d862-6ed4-4dc3-a58f-65779ac8d0b8 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:59.363038 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af0d6d-7590-436c-9cd2-2ccbe8fb6ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:59.364403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af0d6d-7590-436c-9cd2-2ccbe8fb6ead None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:59.364906 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af0d6d-7590-436c-9cd2-2ccbe8fb6ead None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:59.365408 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af0d6d-7590-436c-9cd2-2ccbe8fb6ead None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:59.387456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af0d6d-7590-436c-9cd2-2ccbe8fb6ead None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:40:59.387935 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465699f-d7e8-47ec-ae57-67805c5ed7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:59.389134 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465699f-d7e8-47ec-ae57-67805c5ed7d0 None admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:59.389621 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465699f-d7e8-47ec-ae57-67805c5ed7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:59.390161 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465699f-d7e8-47ec-ae57-67805c5ed7d0 None admin] PATH_INFO: `/v3/system/users/68ac7291f5b24ecfbf7b31b93607d1a9/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:59.392697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465699f-d7e8-47ec-ae57-67805c5ed7d0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=68ac7291f5b24ecfbf7b31b93607d1a9, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:59.394466 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465699f-d7e8-47ec-ae57-67805c5ed7d0 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:59.410227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-b2af0d6d-7590-436c-9cd2-2ccbe8fb6ead None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:40:59.410227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:40:59.410227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:40:59.410227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:40:59.410227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:40:59.410227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:40:59.410227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:40:59.410227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:40:59.410227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:40:59.410227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:40:59.410227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:40:59.410227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:40:59.410227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:40:59.410227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:40:59.410227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:40:59.410227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:40:59.412033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:40:59.412033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:40:59.412033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:40:59.412033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:40:59.412033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:40:59.412033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:40:59.412033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:40:59.412033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:40:59.412033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:40:59.412033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:40:59.412033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:40:59.412033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:40:59.412033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:40:59.412033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:40:59.412033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:40:59.412033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:40:59.413840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:40:59.413840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:40:59.413840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:40:59.413840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:40:59.413840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:40:59.413840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:40:59.413840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:40:59.413840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:40:59.413840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:40:59.413840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:40:59.413840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:40:59.413840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:40:59.413840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 152/313] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:40:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:40:59.415974 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 161/313] 10.222.0.78 () {58 vars in 1334 bytes} [Wed May 25 14:40:59 2022] PUT /identity/v3/system/users/68ac7291f5b24ecfbf7b31b93607d1a9/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:40:59.420313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af0d6d-7590-436c-9cd2-2ccbe8fb6ead None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:59.627591 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af0d6d-7590-436c-9cd2-2ccbe8fb6ead None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:59.669140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c7b61-9a1d-4834-b980-f8b8383c3b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:59.669719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c7b61-9a1d-4834-b980-f8b8383c3b96 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:59.670036 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c7b61-9a1d-4834-b980-f8b8383c3b96 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:59.670314 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c7b61-9a1d-4834-b980-f8b8383c3b96 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:59.671095 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c7b61-9a1d-4834-b980-f8b8383c3b96 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:59.671407 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c7b61-9a1d-4834-b980-f8b8383c3b96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:59.672836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c7b61-9a1d-4834-b980-f8b8383c3b96 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:59.675132 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:59.686118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 153/314] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:40:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:59.695216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465699f-d7e8-47ec-ae57-67805c5ed7d0 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:59.864688 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465699f-d7e8-47ec-ae57-67805c5ed7d0 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:59.930267 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8c504-823c-451d-bf2f-c6066e84cf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:59.931756 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8c504-823c-451d-bf2f-c6066e84cf75 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:59.932235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8c504-823c-451d-bf2f-c6066e84cf75 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:59.932789 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8c504-823c-451d-bf2f-c6066e84cf75 None admin] PATH_INFO: `/v3/users/9e130e15017b49498c39de87c26d58e4` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:59.935200 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8c504-823c-451d-bf2f-c6066e84cf75 None admin] RBAC: Authorizing `identity:get_user(user_id=9e130e15017b49498c39de87c26d58e4)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:59.937881 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8c504-823c-451d-bf2f-c6066e84cf75 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:59.941331 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 162/315] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:40:59 2022] GET /identity/v3/users/9e130e15017b49498c39de87c26d58e4 => generated 297 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:59.949918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c7b61-9a1d-4834-b980-f8b8383c3b96 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:00.106883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c7b61-9a1d-4834-b980-f8b8383c3b96 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:00.183341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322eb2b-2f69-46cb-ae10-ecf92578078d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:00.184108 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322eb2b-2f69-46cb-ae10-ecf92578078d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:00.185005 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322eb2b-2f69-46cb-ae10-ecf92578078d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:00.185005 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322eb2b-2f69-46cb-ae10-ecf92578078d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:00.312039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:00.327448 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f322eb2b-2f69-46cb-ae10-ecf92578078d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:00.331413 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f322eb2b-2f69-46cb-ae10-ecf92578078d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:00.489958 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 154/316] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:40:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:41:00.495625 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8c504-823c-451d-bf2f-c6066e84cf75 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:00.536899 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:00.645313 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8c504-823c-451d-bf2f-c6066e84cf75 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:00.685221 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59eab6-a7f4-488b-88ec-4a4aee4c626f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:00.686140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59eab6-a7f4-488b-88ec-4a4aee4c626f None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:00.686426 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59eab6-a7f4-488b-88ec-4a4aee4c626f None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:00.686702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59eab6-a7f4-488b-88ec-4a4aee4c626f None admin] PATH_INFO: `/v3/projects/212e33252b164b33ae38cf0925858a09` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:00.688039 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c59eab6-a7f4-488b-88ec-4a4aee4c626f None admin] RBAC: Authorizing `identity:get_project(project_id=212e33252b164b33ae38cf0925858a09)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:00.690097 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c59eab6-a7f4-488b-88ec-4a4aee4c626f None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:00.692240 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 163/317] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:41:00 2022] GET /identity/v3/projects/212e33252b164b33ae38cf0925858a09 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:00.698062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322eb2b-2f69-46cb-ae10-ecf92578078d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:00.817373 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:00.824764 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322eb2b-2f69-46cb-ae10-ecf92578078d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:00.868964 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c05b1-f0d3-4224-b609-c79adac0f728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:00.869968 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c05b1-f0d3-4224-b609-c79adac0f728 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:00.870533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c05b1-f0d3-4224-b609-c79adac0f728 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:00.870948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c05b1-f0d3-4224-b609-c79adac0f728 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:00.872309 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c05b1-f0d3-4224-b609-c79adac0f728 None admin] RBAC: Adding query filter params (role.id=ebb0d7a0f606450a8dbfee510b7b0b80, scope.project.id=212e33252b164b33ae38cf0925858a09, user.id=9e130e15017b49498c39de87c26d58e4) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:00.872784 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c05b1-f0d3-4224-b609-c79adac0f728 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:00.875165 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c05b1-f0d3-4224-b609-c79adac0f728 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:00.887754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 155/318] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:41:00 2022] GET /identity/v3/role_assignments?user.id=9e130e15017b49498c39de87c26d58e4&scope.project.id=212e33252b164b33ae38cf0925858a09&role.id=ebb0d7a0f606450a8dbfee510b7b0b80 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:01.453402 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109219e-e76c-4801-a209-515a4d4d8c22 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:01.453935 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109219e-e76c-4801-a209-515a4d4d8c22 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:01.453935 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109219e-e76c-4801-a209-515a4d4d8c22 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:01.455654 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 164/319] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:01.461334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d4ee0-2b50-4dc7-88f1-c6d290171360 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:01.461657 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d4ee0-2b50-4dc7-88f1-c6d290171360 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:01.461903 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d4ee0-2b50-4dc7-88f1-c6d290171360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:01.480264 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-ca0d4ee0-2b50-4dc7-88f1-c6d290171360 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:01.481500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ca0d4ee0-2b50-4dc7-88f1-c6d290171360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:01.557029 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da3985-4fdb-4215-8ef9-5d0b2e96d247 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:01.557394 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da3985-4fdb-4215-8ef9-5d0b2e96d247 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:01.557584 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da3985-4fdb-4215-8ef9-5d0b2e96d247 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:01.559230 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 165/320] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:41:01.562068 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 156/321] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:01.565780 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3e774-c96a-47bb-acc1-b95de21a5a8b None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:01.566117 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3e774-c96a-47bb-acc1-b95de21a5a8b None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:01.566399 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3e774-c96a-47bb-acc1-b95de21a5a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:01.584957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d4ee0-2b50-4dc7-88f1-c6d290171360 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:01.585148 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-40a3e774-c96a-47bb-acc1-b95de21a5a8b None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:01.586219 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-40a3e774-c96a-47bb-acc1-b95de21a5a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:01.669635 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 166/322] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:41:01.692165 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:01.703367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d4ee0-2b50-4dc7-88f1-c6d290171360 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:01.711564 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3e774-c96a-47bb-acc1-b95de21a5a8b None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:01.747682 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0dc63-5943-4dc3-b223-1e307773623b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:01.748503 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0dc63-5943-4dc3-b223-1e307773623b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:01.748781 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0dc63-5943-4dc3-b223-1e307773623b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:01.749175 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0dc63-5943-4dc3-b223-1e307773623b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:01.751594 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:01.764321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0dc63-5943-4dc3-b223-1e307773623b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:01.772178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-ffb0dc63-5943-4dc3-b223-1e307773623b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:41:01.772178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:01.772178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:01.772178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:01.772178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:01.772178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:01.772178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:01.772178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:01.772178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:01.772178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:01.772178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:01.772178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:01.772178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:01.772178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:01.772178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:01.772178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:01.774584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:01.774584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:01.774584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:01.774584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:01.774584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:01.774584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:01.774584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:01.774584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:01.774584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:01.774584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:01.774584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:01.774584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:01.774584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:01.774584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:01.774584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:01.774584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:01.776181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:01.776181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:01.776181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:01.776181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:01.776181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:01.776181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:01.776181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:01.776181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:01.776181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:01.776181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:01.776181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:41:01.776181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:01.776181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 157/323] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:41:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:41:01.779335 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0dc63-5943-4dc3-b223-1e307773623b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:01.850348 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3e774-c96a-47bb-acc1-b95de21a5a8b None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:01.888172 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b5ab4-888d-48f7-bbf3-0243546ec3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:01.888977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b5ab4-888d-48f7-bbf3-0243546ec3e5 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:01.889257 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b5ab4-888d-48f7-bbf3-0243546ec3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:01.889533 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b5ab4-888d-48f7-bbf3-0243546ec3e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:01.890776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b5ab4-888d-48f7-bbf3-0243546ec3e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:01.892329 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b5ab4-888d-48f7-bbf3-0243546ec3e5 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:01.894204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 167/324] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:01.901441 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b5ab4-888d-48f7-bbf3-0243546ec3e5 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:01.958950 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0dc63-5943-4dc3-b223-1e307773623b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:01.998393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcecad1-0e5b-42e5-bd2f-a29e12474a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:01.998969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcecad1-0e5b-42e5-bd2f-a29e12474a47 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:01.999255 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcecad1-0e5b-42e5-bd2f-a29e12474a47 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:01.999533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcecad1-0e5b-42e5-bd2f-a29e12474a47 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:02.000289 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcecad1-0e5b-42e5-bd2f-a29e12474a47 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:02.000599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcecad1-0e5b-42e5-bd2f-a29e12474a47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:02.002059 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcecad1-0e5b-42e5-bd2f-a29e12474a47 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:02.010988 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 158/325] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:41:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:02.017396 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcecad1-0e5b-42e5-bd2f-a29e12474a47 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:02.033832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:02.066983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:02.074382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b5ab4-888d-48f7-bbf3-0243546ec3e5 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:02.134890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efea49-9f41-4d4a-abba-35467f4836c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:02.135558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efea49-9f41-4d4a-abba-35467f4836c0 None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:02.135843 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efea49-9f41-4d4a-abba-35467f4836c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:02.136107 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efea49-9f41-4d4a-abba-35467f4836c0 None admin] PATH_INFO: `/v3/groups` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:02.136439 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcecad1-0e5b-42e5-bd2f-a29e12474a47 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:02.136816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efea49-9f41-4d4a-abba-35467f4836c0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:02.138370 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efea49-9f41-4d4a-abba-35467f4836c0 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:02.156950 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 168/326] 10.222.0.78 () {60 vars in 1134 bytes} [Wed May 25 14:41:01 2022] POST /identity/v3/groups => generated 226 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:02.188175 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdc3cc-8faa-4579-9e21-7bea5ce55302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:02.189723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdc3cc-8faa-4579-9e21-7bea5ce55302 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:02.190044 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdc3cc-8faa-4579-9e21-7bea5ce55302 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:02.190325 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdc3cc-8faa-4579-9e21-7bea5ce55302 None admin] PATH_INFO: `/v3/users/601eb237a1074978b1b65807d2045288` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:02.191770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdc3cc-8faa-4579-9e21-7bea5ce55302 None admin] RBAC: Authorizing `identity:get_user(user_id=601eb237a1074978b1b65807d2045288)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:02.193320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdc3cc-8faa-4579-9e21-7bea5ce55302 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:02.195420 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 159/327] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:41:02 2022] GET /identity/v3/users/601eb237a1074978b1b65807d2045288 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:02.201254 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efea49-9f41-4d4a-abba-35467f4836c0 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:02.361499 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efea49-9f41-4d4a-abba-35467f4836c0 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:02.405536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b724c2-b609-4cb7-a733-cdf1f3ecce8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:02.406258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b724c2-b609-4cb7-a733-cdf1f3ecce8f None admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:02.406537 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b724c2-b609-4cb7-a733-cdf1f3ecce8f None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:02.406808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b724c2-b609-4cb7-a733-cdf1f3ecce8f None admin] PATH_INFO: `/v3/system/users/601eb237a1074978b1b65807d2045288/roles/ebb0d7a0f606450a8dbfee510b7b0b80` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:02.408998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b724c2-b609-4cb7-a733-cdf1f3ecce8f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=601eb237a1074978b1b65807d2045288, role_id=ebb0d7a0f606450a8dbfee510b7b0b80)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:02.410472 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b724c2-b609-4cb7-a733-cdf1f3ecce8f None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:02.418895 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 169/328] 10.222.0.78 () {58 vars in 1334 bytes} [Wed May 25 14:41:02 2022] PUT /identity/v3/system/users/601eb237a1074978b1b65807d2045288/roles/ebb0d7a0f606450a8dbfee510b7b0b80 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:03.037757 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc19592-9d03-496e-84db-11d32dbc02f6 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:03.038628 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc19592-9d03-496e-84db-11d32dbc02f6 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:03.038976 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc19592-9d03-496e-84db-11d32dbc02f6 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:03.041881 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 160/329] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:03.052241 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f485250-a1c2-47db-8566-8af03ef75d7d None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:03.053252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f485250-a1c2-47db-8566-8af03ef75d7d None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:03.053890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f485250-a1c2-47db-8566-8af03ef75d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:03.078939 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-9f485250-a1c2-47db-8566-8af03ef75d7d None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:03.081114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-9f485250-a1c2-47db-8566-8af03ef75d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:03.107162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:03.184091 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 170/330] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:03.223092 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc19592-9d03-496e-84db-11d32dbc02f6 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:03.412432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:03.456171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc19592-9d03-496e-84db-11d32dbc02f6 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:03.496388 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c963d015-3321-45d7-b93e-c05905a6c643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:03.497195 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963d015-3321-45d7-b93e-c05905a6c643 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:03.497479 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963d015-3321-45d7-b93e-c05905a6c643 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:03.497759 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963d015-3321-45d7-b93e-c05905a6c643 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:03.510039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c963d015-3321-45d7-b93e-c05905a6c643 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:03.518931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-c963d015-3321-45d7-b93e-c05905a6c643 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:41:03.518931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:03.518931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:03.518931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:03.518931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:03.518931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:03.518931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:03.518931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:03.518931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:03.518931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:03.518931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:03.518931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:03.518931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:03.518931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:03.518931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:03.518931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:03.520564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:03.520564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:03.520564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:03.520564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:03.520564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:03.520564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:03.520564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:03.520564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:03.520564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:03.520564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:03.520564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:03.520564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:03.520564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:03.520564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:03.520564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:03.520564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:03.522031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:03.522031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:03.522031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:03.522031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:03.522031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:03.522031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:03.522031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:03.522031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:03.522031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:03.522031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:03.522031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:41:03.522031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:03.522031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 161/331] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:41:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:03.527318 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f485250-a1c2-47db-8566-8af03ef75d7d None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:03.762452 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f485250-a1c2-47db-8566-8af03ef75d7d None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:03.838772 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ac3fa-5528-4ea2-be01-c318a9dcc7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:03.840323 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ac3fa-5528-4ea2-be01-c318a9dcc7ce None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:03.841072 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ac3fa-5528-4ea2-be01-c318a9dcc7ce None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:03.841874 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ac3fa-5528-4ea2-be01-c318a9dcc7ce None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:03.843528 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ac3fa-5528-4ea2-be01-c318a9dcc7ce None admin] RBAC: Adding query filter params (name=member) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:03.844385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ac3fa-5528-4ea2-be01-c318a9dcc7ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:03.847877 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ac3fa-5528-4ea2-be01-c318a9dcc7ce None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:03.862132 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 171/332] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:41:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:03.868366 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c963d015-3321-45d7-b93e-c05905a6c643 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:04.071511 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c963d015-3321-45d7-b93e-c05905a6c643 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:04.120923 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08550f-9b23-4e79-9642-a199aa7f3def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:04.121866 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08550f-9b23-4e79-9642-a199aa7f3def None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:04.122205 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08550f-9b23-4e79-9642-a199aa7f3def None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:04.122491 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08550f-9b23-4e79-9642-a199aa7f3def None admin] PATH_INFO: `/v3/users/68ac7291f5b24ecfbf7b31b93607d1a9` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:04.124055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08550f-9b23-4e79-9642-a199aa7f3def None admin] RBAC: Authorizing `identity:get_user(user_id=68ac7291f5b24ecfbf7b31b93607d1a9)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:04.125586 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08550f-9b23-4e79-9642-a199aa7f3def None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:04.131040 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 162/333] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:41:03 2022] GET /identity/v3/users/68ac7291f5b24ecfbf7b31b93607d1a9 => generated 293 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:04.135000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ac3fa-5528-4ea2-be01-c318a9dcc7ce None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:04.156367 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:04.271726 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ac3fa-5528-4ea2-be01-c318a9dcc7ce None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:04.326650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dbd268-0614-4a70-9984-fbc21367d18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:04.327576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbd268-0614-4a70-9984-fbc21367d18b None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:04.328146 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbd268-0614-4a70-9984-fbc21367d18b None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:04.328615 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbd268-0614-4a70-9984-fbc21367d18b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:04.330854 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dbd268-0614-4a70-9984-fbc21367d18b None admin] RBAC: Adding query filter params (role.id=a3da3c62738d4790b9dac512b1568b52, user.id=68ac7291f5b24ecfbf7b31b93607d1a9, scope.system=all) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:04.331478 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dbd268-0614-4a70-9984-fbc21367d18b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:04.334472 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dbd268-0614-4a70-9984-fbc21367d18b None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:04.354455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 172/334] 10.222.0.78 () {56 vars in 1308 bytes} [Wed May 25 14:41:04 2022] GET /identity/v3/role_assignments?user.id=68ac7291f5b24ecfbf7b31b93607d1a9&scope.system=all&role.id=a3da3c62738d4790b9dac512b1568b52 => generated 510 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:06.413759 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48344d36-fe9c-4e1d-8cb8-6365affd8967 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:06.415310 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48344d36-fe9c-4e1d-8cb8-6365affd8967 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:06.416137 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48344d36-fe9c-4e1d-8cb8-6365affd8967 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:06.419503 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 163/335] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:41:06.427981 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69c016-0018-4804-a73d-f0815d5adb37 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:06.429475 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69c016-0018-4804-a73d-f0815d5adb37 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:06.430242 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69c016-0018-4804-a73d-f0815d5adb37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:06.471362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-cc69c016-0018-4804-a73d-f0815d5adb37 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:06.474001 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-cc69c016-0018-4804-a73d-f0815d5adb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:06.612776 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdea90-aac9-433c-8db0-8926388c0ef6 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:06.613703 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdea90-aac9-433c-8db0-8926388c0ef6 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:06.614500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdea90-aac9-433c-8db0-8926388c0ef6 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:06.617338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 164/336] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:06.629437 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aea699-99b1-4894-8e1d-336ea2ef00ee None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:06.630369 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aea699-99b1-4894-8e1d-336ea2ef00ee None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:06.631119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aea699-99b1-4894-8e1d-336ea2ef00ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:06.675627 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-77aea699-99b1-4894-8e1d-336ea2ef00ee None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:06.679856 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-77aea699-99b1-4894-8e1d-336ea2ef00ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:06.679856 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:06.691940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 173/337] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:06.755029 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69c016-0018-4804-a73d-f0815d5adb37 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:06.758353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:06.818109 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 165/338] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:41:06.859183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aea699-99b1-4894-8e1d-336ea2ef00ee None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:06.974134 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69c016-0018-4804-a73d-f0815d5adb37 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:07.038491 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba050de0-5611-4a7a-a4b2-b65cc9644c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:07.040415 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba050de0-5611-4a7a-a4b2-b65cc9644c6b None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:07.040710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba050de0-5611-4a7a-a4b2-b65cc9644c6b None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:07.040972 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba050de0-5611-4a7a-a4b2-b65cc9644c6b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:07.065409 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba050de0-5611-4a7a-a4b2-b65cc9644c6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:07.075278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-ba050de0-5611-4a7a-a4b2-b65cc9644c6b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:41:07.075278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:07.075278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:07.075278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:07.075278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:07.075278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:07.075278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:07.075278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:07.075278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:07.075278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:07.075278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:07.075278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:07.075278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:07.075278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:07.075278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:07.075278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:07.077207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:07.077207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:07.077207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:07.077207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:07.077207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:07.077207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:07.077207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:07.077207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:07.077207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:07.077207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:07.077207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:07.077207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:07.077207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:07.077207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:07.077207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:07.077207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:07.079728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:07.079728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:07.079728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:07.079728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:07.079728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:07.079728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:07.079728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:07.079728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:07.079728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:07.079728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:07.079728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:41:07.079728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:07.079728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 174/339] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:41:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:41:07.085963 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba050de0-5611-4a7a-a4b2-b65cc9644c6b None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:07.121156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aea699-99b1-4894-8e1d-336ea2ef00ee None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:07.220497 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c52614-123b-470e-9166-1a082973521b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:07.220761 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:07.221769 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c52614-123b-470e-9166-1a082973521b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:07.222227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c52614-123b-470e-9166-1a082973521b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:07.222527 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c52614-123b-470e-9166-1a082973521b None admin] PATH_INFO: `/v3/roles/member` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:07.241138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:07.249400 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c52614-123b-470e-9166-1a082973521b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:07.265465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-62c52614-123b-470e-9166-1a082973521b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:41:07.265465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:07.265465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:07.265465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:07.265465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:07.265465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:07.265465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:07.265465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:07.265465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:07.265465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:07.265465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:07.265465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:07.265465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:07.265465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:07.265465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:07.265465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:07.268972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:07.268972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:07.268972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:07.268972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:07.268972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:07.268972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:07.268972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:07.268972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:07.268972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:07.268972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:07.268972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:07.268972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:07.268972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:07.268972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:07.268972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:07.268972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:07.271719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:07.271719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:07.271719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:07.271719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:07.271719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:07.271719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:07.271719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:07.271719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:07.271719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:07.271719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:07.271719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:41:07.271719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:07.271719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 166/340] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:41:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:41:07.278183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c52614-123b-470e-9166-1a082973521b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:07.321226 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba050de0-5611-4a7a-a4b2-b65cc9644c6b None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:07.427962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f29f22-6cbf-4f3a-b188-7f5fc4521aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:07.428665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f29f22-6cbf-4f3a-b188-7f5fc4521aab None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:07.429472 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f29f22-6cbf-4f3a-b188-7f5fc4521aab None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:07.429472 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f29f22-6cbf-4f3a-b188-7f5fc4521aab None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:07.429975 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f29f22-6cbf-4f3a-b188-7f5fc4521aab None admin] RBAC: Adding query filter params (name=reader) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:07.430318 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f29f22-6cbf-4f3a-b188-7f5fc4521aab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:07.431797 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f29f22-6cbf-4f3a-b188-7f5fc4521aab None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:07.449830 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 175/341] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:41:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:07.463272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f29f22-6cbf-4f3a-b188-7f5fc4521aab None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:07.522448 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c52614-123b-470e-9166-1a082973521b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:07.588563 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4eae0-48b7-4641-970d-a37e128fe579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:07.589727 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4eae0-48b7-4641-970d-a37e128fe579 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:07.590395 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4eae0-48b7-4641-970d-a37e128fe579 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:07.591028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4eae0-48b7-4641-970d-a37e128fe579 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:07.592486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4eae0-48b7-4641-970d-a37e128fe579 None admin] RBAC: Adding query filter params (name=member) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:07.593039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4eae0-48b7-4641-970d-a37e128fe579 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:07.596312 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4eae0-48b7-4641-970d-a37e128fe579 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:07.613435 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 167/342] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:41:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:07.615905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeded57-ab39-44dc-9237-a319ddad327e None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:07.616571 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeded57-ab39-44dc-9237-a319ddad327e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:07.617088 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeded57-ab39-44dc-9237-a319ddad327e None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:07.619786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 168/343] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:07.625730 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0aff5-0437-4c54-93b1-26b8f7ba2631 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:07.625730 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0aff5-0437-4c54-93b1-26b8f7ba2631 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:07.625730 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0aff5-0437-4c54-93b1-26b8f7ba2631 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:07.627094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 169/344] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:07.632400 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f29f22-6cbf-4f3a-b188-7f5fc4521aab None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:07.633091 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0aff5-0437-4c54-93b1-26b8f7ba2631 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:07.669397 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:07.699165 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573a036-90ab-4342-bceb-5e12cf49b134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:07.701292 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573a036-90ab-4342-bceb-5e12cf49b134 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:07.701292 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573a036-90ab-4342-bceb-5e12cf49b134 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:07.701619 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573a036-90ab-4342-bceb-5e12cf49b134 None admin] PATH_INFO: `/v3/users/601eb237a1074978b1b65807d2045288` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:07.708331 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6573a036-90ab-4342-bceb-5e12cf49b134 None admin] RBAC: Authorizing `identity:get_user(user_id=601eb237a1074978b1b65807d2045288)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:07.711162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6573a036-90ab-4342-bceb-5e12cf49b134 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:07.714729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 176/345] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:41:07 2022] GET /identity/v3/users/601eb237a1074978b1b65807d2045288 => generated 293 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:07.716976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d43104-b866-4533-be1a-e7c4a6ffe909 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:07.717453 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d43104-b866-4533-be1a-e7c4a6ffe909 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:07.717955 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d43104-b866-4533-be1a-e7c4a6ffe909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:07.734587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:07.743905 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-b7d43104-b866-4533-be1a-e7c4a6ffe909 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:07.746279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-b7d43104-b866-4533-be1a-e7c4a6ffe909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:07.822915 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0aff5-0437-4c54-93b1-26b8f7ba2631 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:07.875479 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 177/346] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:07.877406 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bf794-061c-40ba-96ac-89cdfe539fe7 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:07.877746 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bf794-061c-40ba-96ac-89cdfe539fe7 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:07.878060 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bf794-061c-40ba-96ac-89cdfe539fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:07.888793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ec42e-739f-4e0b-a4e3-ef851f4073f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:07.890375 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ec42e-739f-4e0b-a4e3-ef851f4073f8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:07.891116 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ec42e-739f-4e0b-a4e3-ef851f4073f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:07.892441 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ec42e-739f-4e0b-a4e3-ef851f4073f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:07.909363 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-cd3bf794-061c-40ba-96ac-89cdfe539fe7 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:07.910613 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-cd3bf794-061c-40ba-96ac-89cdfe539fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:08.003938 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ec42e-739f-4e0b-a4e3-ef851f4073f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:08.005721 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ec42e-739f-4e0b-a4e3-ef851f4073f8 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:08.030793 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 178/347] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:08.033201 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cdb18-b349-4374-a09b-12a06fe1516f None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:08.033484 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cdb18-b349-4374-a09b-12a06fe1516f None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:08.033761 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cdb18-b349-4374-a09b-12a06fe1516f None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:08.038503 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 179/348] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:08.038503 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cdb18-b349-4374-a09b-12a06fe1516f None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:08.112260 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:08.153247 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:08.205376 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 170/349] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:41:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:41:08.206956 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ec42e-739f-4e0b-a4e3-ef851f4073f8 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:08.246686 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cdb18-b349-4374-a09b-12a06fe1516f None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:08.321757 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5dcaae-3785-45da-ac50-f42eda60fc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:08.322957 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dcaae-3785-45da-ac50-f42eda60fc5b None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:08.323457 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dcaae-3785-45da-ac50-f42eda60fc5b None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:08.323949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dcaae-3785-45da-ac50-f42eda60fc5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:08.325582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5dcaae-3785-45da-ac50-f42eda60fc5b None admin] RBAC: Adding query filter params (role.id=ebb0d7a0f606450a8dbfee510b7b0b80, user.id=601eb237a1074978b1b65807d2045288, scope.system=all) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:08.332606 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5dcaae-3785-45da-ac50-f42eda60fc5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:08.332606 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5dcaae-3785-45da-ac50-f42eda60fc5b None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:08.354892 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 180/350] 10.222.0.78 () {56 vars in 1308 bytes} [Wed May 25 14:41:08 2022] GET /identity/v3/role_assignments?user.id=601eb237a1074978b1b65807d2045288&scope.system=all&role.id=ebb0d7a0f606450a8dbfee510b7b0b80 => generated 510 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:08.356237 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ec42e-739f-4e0b-a4e3-ef851f4073f8 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:08.357925 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be02df-c854-476c-ae22-9e20fba48c9b None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:08.358787 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be02df-c854-476c-ae22-9e20fba48c9b None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:08.359556 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be02df-c854-476c-ae22-9e20fba48c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:08.393519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-76be02df-c854-476c-ae22-9e20fba48c9b None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:08.396380 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-76be02df-c854-476c-ae22-9e20fba48c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:08.398340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865d5e4-e971-4388-9d22-48e31351d622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:08.399250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865d5e4-e971-4388-9d22-48e31351d622 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:08.399601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865d5e4-e971-4388-9d22-48e31351d622 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:08.399882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865d5e4-e971-4388-9d22-48e31351d622 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:08.413870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865d5e4-e971-4388-9d22-48e31351d622 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:08.423574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-3865d5e4-e971-4388-9d22-48e31351d622 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:41:08.423574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:08.423574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:08.423574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:08.423574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:08.423574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:08.423574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:08.423574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:08.423574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:08.423574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:08.423574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:08.423574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:08.423574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:08.423574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:08.423574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:08.423574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:08.424754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:08.424754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:08.424754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:08.424754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:08.424754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:08.424754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:08.424754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:08.424754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:08.424754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:08.424754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:08.424754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:08.424754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:08.424754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:08.424754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:08.424754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:08.424754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:08.425758 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:08.425758 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:08.425758 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:08.425758 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:08.425758 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:08.425758 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:08.425758 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:08.425758 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:08.425758 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:08.425758 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:08.425758 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:41:08.425758 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:08.425758 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 171/351] 10.222.0.78 () {56 vars in 1114 bytes} [Wed May 25 14:41:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:08.430797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865d5e4-e971-4388-9d22-48e31351d622 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:08.460487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:08.534072 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 181/352] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:08.535498 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be02df-c854-476c-ae22-9e20fba48c9b None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:08.565937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865d5e4-e971-4388-9d22-48e31351d622 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:08.608798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4b7b4-96b6-4c54-a163-5f14cec06c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:08.611758 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4b7b4-96b6-4c54-a163-5f14cec06c47 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:08.612326 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4b7b4-96b6-4c54-a163-5f14cec06c47 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:08.612763 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4b7b4-96b6-4c54-a163-5f14cec06c47 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:08.626578 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e4b7b4-96b6-4c54-a163-5f14cec06c47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:08.634984 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:08.637514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-61e4b7b4-96b6-4c54-a163-5f14cec06c47 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:41:08.637514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:08.637514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:08.637514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:08.637514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:08.637514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:08.637514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:08.637514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:08.637514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:08.637514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:08.637514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:08.637514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:08.637514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:08.637514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:08.637514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:08.637514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:08.638658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:08.638658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:08.638658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:08.638658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:08.638658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:08.638658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:08.638658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:08.638658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:08.638658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:08.638658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:08.638658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:08.638658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:08.638658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:08.638658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:08.638658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:08.638658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:08.639634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:08.639634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:08.639634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:08.639634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:08.639634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:08.639634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:08.639634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:08.639634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:08.639634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:08.639634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:08.639634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:41:08.639634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:08.642476 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 172/353] 10.222.0.78 () {56 vars in 1114 bytes} [Wed May 25 14:41:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:08.644114 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4b7b4-96b6-4c54-a163-5f14cec06c47 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:08.722140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be02df-c854-476c-ae22-9e20fba48c9b None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:08.777029 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b84b26-1853-461a-8b6c-da05b25667e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:08.777029 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b84b26-1853-461a-8b6c-da05b25667e2 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:08.777029 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b84b26-1853-461a-8b6c-da05b25667e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:08.777573 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b84b26-1853-461a-8b6c-da05b25667e2 None admin] PATH_INFO: `/v3/projects/9799d178c2a042baa9963766f953c6a6` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:08.779341 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b84b26-1853-461a-8b6c-da05b25667e2 None admin] RBAC: Authorizing `identity:get_project(project_id=9799d178c2a042baa9963766f953c6a6)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:08.780840 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b84b26-1853-461a-8b6c-da05b25667e2 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:08.783305 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 182/354] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:41:08 2022] GET /identity/v3/projects/9799d178c2a042baa9963766f953c6a6 => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:08.784119 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b84b26-1853-461a-8b6c-da05b25667e2 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:08.824138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:08.865406 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4b7b4-96b6-4c54-a163-5f14cec06c47 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:08.933692 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196eb0e-0415-48d6-8e6d-0d6df88aba4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:08.934501 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196eb0e-0415-48d6-8e6d-0d6df88aba4e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:08.934784 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196eb0e-0415-48d6-8e6d-0d6df88aba4e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:08.935060 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196eb0e-0415-48d6-8e6d-0d6df88aba4e None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:08.935822 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196eb0e-0415-48d6-8e6d-0d6df88aba4e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:08.936126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196eb0e-0415-48d6-8e6d-0d6df88aba4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:08.937557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196eb0e-0415-48d6-8e6d-0d6df88aba4e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:08.949824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b84b26-1853-461a-8b6c-da05b25667e2 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:08.951322 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 173/355] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:41:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:08.951960 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196eb0e-0415-48d6-8e6d-0d6df88aba4e None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:08.972156 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:09.011215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e518e2-8e16-4415-8b1d-112efb2784d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:09.011996 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e518e2-8e16-4415-8b1d-112efb2784d3 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:09.012272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e518e2-8e16-4415-8b1d-112efb2784d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:09.012541 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e518e2-8e16-4415-8b1d-112efb2784d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:09.026824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e518e2-8e16-4415-8b1d-112efb2784d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:09.035289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-61e518e2-8e16-4415-8b1d-112efb2784d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:41:09.035289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:09.035289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:09.035289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:09.035289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:09.035289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:09.035289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:09.035289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:09.035289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:09.035289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:09.035289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:09.035289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:09.035289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:09.035289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:09.035289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:09.035289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:09.036366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:09.036366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:09.036366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:09.036366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:09.036366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:09.036366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:09.036366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:09.036366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:09.036366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:09.036366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:09.036366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:09.036366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:09.036366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:09.036366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:09.036366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:09.036366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:09.037162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:09.037162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:09.037162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:09.037162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:09.037162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:09.037162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:09.037162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:09.037162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:09.037162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:09.037162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:09.037162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:41:09.037162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:09.037162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 183/356] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:41:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:09.038748 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb10ce-d964-4157-a69c-7e01c3c6000a None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:09.038748 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb10ce-d964-4157-a69c-7e01c3c6000a None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:09.038888 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb10ce-d964-4157-a69c-7e01c3c6000a None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:09.040507 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 184/357] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:09.040964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb10ce-d964-4157-a69c-7e01c3c6000a None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:09.114167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196eb0e-0415-48d6-8e6d-0d6df88aba4e None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:09.158883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247bc2e-c53f-4c16-b9aa-25ffe55b876c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:09.160381 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247bc2e-c53f-4c16-b9aa-25ffe55b876c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:09.160671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247bc2e-c53f-4c16-b9aa-25ffe55b876c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:09.160949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247bc2e-c53f-4c16-b9aa-25ffe55b876c None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:09.161719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b247bc2e-c53f-4c16-b9aa-25ffe55b876c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:09.162111 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b247bc2e-c53f-4c16-b9aa-25ffe55b876c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:09.163869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b247bc2e-c53f-4c16-b9aa-25ffe55b876c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:09.175972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 174/358] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:41:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:09.175972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247bc2e-c53f-4c16-b9aa-25ffe55b876c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:09.181006 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:09.199297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb10ce-d964-4157-a69c-7e01c3c6000a None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:09.243644 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74526328-ad40-4a9e-9cd7-91bbccf9e017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:09.243644 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74526328-ad40-4a9e-9cd7-91bbccf9e017 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:09.244161 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74526328-ad40-4a9e-9cd7-91bbccf9e017 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:09.244161 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74526328-ad40-4a9e-9cd7-91bbccf9e017 None admin] PATH_INFO: `/v3/groups/59d498c7d42543ad9a0d6568a293c893` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:09.252197 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74526328-ad40-4a9e-9cd7-91bbccf9e017 None admin] RBAC: Authorizing `identity:get_group(group_id=59d498c7d42543ad9a0d6568a293c893)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:09.253711 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74526328-ad40-4a9e-9cd7-91bbccf9e017 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:09.255797 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 185/359] 10.222.0.78 () {56 vars in 1180 bytes} [Wed May 25 14:41:09 2022] GET /identity/v3/groups/59d498c7d42543ad9a0d6568a293c893 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:09.257715 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74526328-ad40-4a9e-9cd7-91bbccf9e017 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:09.304398 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:09.336253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247bc2e-c53f-4c16-b9aa-25ffe55b876c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:09.389315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c342dd-de46-48ed-a9df-ac2bae4e5e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:09.389995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c342dd-de46-48ed-a9df-ac2bae4e5e29 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:09.390399 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c342dd-de46-48ed-a9df-ac2bae4e5e29 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:09.390682 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c342dd-de46-48ed-a9df-ac2bae4e5e29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:09.406459 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74526328-ad40-4a9e-9cd7-91bbccf9e017 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:09.455393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759c232-b3b1-40a5-9c37-e76c151ccd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:09.456169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759c232-b3b1-40a5-9c37-e76c151ccd68 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:09.456572 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759c232-b3b1-40a5-9c37-e76c151ccd68 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:09.456968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759c232-b3b1-40a5-9c37-e76c151ccd68 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:09.457875 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759c232-b3b1-40a5-9c37-e76c151ccd68 None admin] RBAC: Adding query filter params (name=member) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:09.458334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759c232-b3b1-40a5-9c37-e76c151ccd68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:09.459990 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759c232-b3b1-40a5-9c37-e76c151ccd68 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:09.475800 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 186/360] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:41:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:09.478207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2241a-763c-46f4-8535-703564bdfe3a None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:09.478712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2241a-763c-46f4-8535-703564bdfe3a None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:09.479083 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2241a-763c-46f4-8535-703564bdfe3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:09.502353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c342dd-de46-48ed-a9df-ac2bae4e5e29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:09.502684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-9ac2241a-763c-46f4-8535-703564bdfe3a None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:09.503849 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c342dd-de46-48ed-a9df-ac2bae4e5e29 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:09.503958 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-9ac2241a-763c-46f4-8535-703564bdfe3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:09.508339 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:09.615906 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 187/361] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:09.616872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2241a-763c-46f4-8535-703564bdfe3a None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:09.651035 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:09.682120 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 175/362] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:41:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:41:09.683161 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c342dd-de46-48ed-a9df-ac2bae4e5e29 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:09.761561 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2241a-763c-46f4-8535-703564bdfe3a None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:09.807287 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bcbe63-7cfb-408c-83d3-0b0e044678b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:09.808330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcbe63-7cfb-408c-83d3-0b0e044678b4 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:09.808816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcbe63-7cfb-408c-83d3-0b0e044678b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:09.809212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcbe63-7cfb-408c-83d3-0b0e044678b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:09.851518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c342dd-de46-48ed-a9df-ac2bae4e5e29 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:09.855158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:09.930930 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a5bdf-b900-4179-a8df-0be7deea7284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:09.932130 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a5bdf-b900-4179-a8df-0be7deea7284 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:09.932621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a5bdf-b900-4179-a8df-0be7deea7284 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:09.932961 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a5bdf-b900-4179-a8df-0be7deea7284 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:09.934544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a5bdf-b900-4179-a8df-0be7deea7284 None admin] RBAC: Adding query filter params (group.id=59d498c7d42543ad9a0d6568a293c893, role.id=a3da3c62738d4790b9dac512b1568b52, scope.project.id=9799d178c2a042baa9963766f953c6a6) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:09.934980 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a5bdf-b900-4179-a8df-0be7deea7284 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:09.937516 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a5bdf-b900-4179-a8df-0be7deea7284 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:09.954482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 176/363] 10.222.0.78 () {56 vars in 1376 bytes} [Wed May 25 14:41:09 2022] GET /identity/v3/role_assignments?group.id=59d498c7d42543ad9a0d6568a293c893&scope.project.id=9799d178c2a042baa9963766f953c6a6&role.id=a3da3c62738d4790b9dac512b1568b52 => generated 260 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:09.954482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a5bdf-b900-4179-a8df-0be7deea7284 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:09.954819 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bcbe63-7cfb-408c-83d3-0b0e044678b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:09.954819 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bcbe63-7cfb-408c-83d3-0b0e044678b4 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:10.000014 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:10.148312 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 188/364] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:41:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:41:10.148748 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a5bdf-b900-4179-a8df-0be7deea7284 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:10.154048 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bcbe63-7cfb-408c-83d3-0b0e044678b4 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:10.213716 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a8b06-67a6-4cfd-868d-ac8cbc2ae8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:10.214759 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a8b06-67a6-4cfd-868d-ac8cbc2ae8f9 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:10.215168 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a8b06-67a6-4cfd-868d-ac8cbc2ae8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:10.215574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a8b06-67a6-4cfd-868d-ac8cbc2ae8f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:10.286684 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:10.288216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bcbe63-7cfb-408c-83d3-0b0e044678b4 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:10.320464 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:10.340260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617c6dc-4c96-433c-b4cb-8c3574fab1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:10.341650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617c6dc-4c96-433c-b4cb-8c3574fab1bb None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:10.342188 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617c6dc-4c96-433c-b4cb-8c3574fab1bb None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:10.342684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617c6dc-4c96-433c-b4cb-8c3574fab1bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:10.362804 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a8b06-67a6-4cfd-868d-ac8cbc2ae8f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:10.364527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617c6dc-4c96-433c-b4cb-8c3574fab1bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:10.365439 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a8b06-67a6-4cfd-868d-ac8cbc2ae8f9 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:10.380850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-d617c6dc-4c96-433c-b4cb-8c3574fab1bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:41:10.380850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:10.380850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:10.380850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:10.380850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:10.380850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:10.380850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:10.380850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:10.380850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:10.380850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:10.380850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:10.380850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:10.380850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:10.380850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:10.380850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:10.380850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:10.381832 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:10.381832 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:10.381832 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:10.381832 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:10.381832 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:10.381832 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:10.381832 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:10.381832 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:10.381832 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:10.381832 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:10.381832 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:10.381832 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:10.381832 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:10.381832 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:10.381832 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:10.381832 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:10.382720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:10.382720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:10.382720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:10.382720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:10.382720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:10.382720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:10.382720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:10.382720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:10.382720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:10.382720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:10.382720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:41:10.382720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:10.383353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 189/365] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:41:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:10.385001 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617c6dc-4c96-433c-b4cb-8c3574fab1bb None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:10.542800 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 177/366] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:41:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:41:10.542800 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a8b06-67a6-4cfd-868d-ac8cbc2ae8f9 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:10.586443 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617c6dc-4c96-433c-b4cb-8c3574fab1bb None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:10.642042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc38910-235b-48ec-afda-3cba854d0925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:10.643522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc38910-235b-48ec-afda-3cba854d0925 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:10.643962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc38910-235b-48ec-afda-3cba854d0925 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:10.644401 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc38910-235b-48ec-afda-3cba854d0925 None admin] PATH_INFO: `/v3/projects/9799d178c2a042baa9963766f953c6a6` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:10.646460 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc38910-235b-48ec-afda-3cba854d0925 None admin] RBAC: Authorizing `identity:get_project(project_id=9799d178c2a042baa9963766f953c6a6)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:10.648998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc38910-235b-48ec-afda-3cba854d0925 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:10.652291 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 190/367] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:41:10 2022] GET /identity/v3/projects/9799d178c2a042baa9963766f953c6a6 => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:10.654332 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc38910-235b-48ec-afda-3cba854d0925 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:10.655937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:10.721736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a8b06-67a6-4cfd-868d-ac8cbc2ae8f9 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:10.754632 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:10.798476 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d96e05-2560-4c78-9f8b-c40d72f1ba8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:10.799241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d96e05-2560-4c78-9f8b-c40d72f1ba8d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:10.799527 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d96e05-2560-4c78-9f8b-c40d72f1ba8d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:10.800463 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d96e05-2560-4c78-9f8b-c40d72f1ba8d None admin] PATH_INFO: `/v3/projects/212e33252b164b33ae38cf0925858a09` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:10.801134 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d96e05-2560-4c78-9f8b-c40d72f1ba8d None admin] RBAC: Authorizing `identity:get_project(project_id=212e33252b164b33ae38cf0925858a09)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:10.802660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d96e05-2560-4c78-9f8b-c40d72f1ba8d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:10.811825 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 178/368] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:41:10 2022] GET /identity/v3/projects/212e33252b164b33ae38cf0925858a09 => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:10.811825 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d96e05-2560-4c78-9f8b-c40d72f1ba8d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:10.838730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc38910-235b-48ec-afda-3cba854d0925 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:10.881642 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d144af-bdcb-40dc-9a44-122390508c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:10.882356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d144af-bdcb-40dc-9a44-122390508c3d None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:10.882677 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d144af-bdcb-40dc-9a44-122390508c3d None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:10.882960 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d144af-bdcb-40dc-9a44-122390508c3d None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:10.883711 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d144af-bdcb-40dc-9a44-122390508c3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:10.884016 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d144af-bdcb-40dc-9a44-122390508c3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:10.885420 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d144af-bdcb-40dc-9a44-122390508c3d None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:10.899576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 191/369] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:41:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:10.900152 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d144af-bdcb-40dc-9a44-122390508c3d None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:11.002500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d96e05-2560-4c78-9f8b-c40d72f1ba8d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:11.047285 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262b72e-10e6-47b8-b7bb-29c77fcaa7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:11.048797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262b72e-10e6-47b8-b7bb-29c77fcaa7df None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:11.049022 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262b72e-10e6-47b8-b7bb-29c77fcaa7df None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:11.049301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262b72e-10e6-47b8-b7bb-29c77fcaa7df None admin] PATH_INFO: `/v3/projects/212e33252b164b33ae38cf0925858a09` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:11.050608 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8262b72e-10e6-47b8-b7bb-29c77fcaa7df None admin] RBAC: Authorizing `identity:get_project(project_id=212e33252b164b33ae38cf0925858a09)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:11.052084 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8262b72e-10e6-47b8-b7bb-29c77fcaa7df None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:11.054057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 179/370] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:41:10 2022] GET /identity/v3/projects/212e33252b164b33ae38cf0925858a09 => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:11.060461 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262b72e-10e6-47b8-b7bb-29c77fcaa7df None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:11.066336 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:11.080025 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d144af-bdcb-40dc-9a44-122390508c3d None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:11.108072 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:11.137081 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706e7de-4511-477a-82f1-a69cc05590b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:11.137947 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706e7de-4511-477a-82f1-a69cc05590b3 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:11.138254 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706e7de-4511-477a-82f1-a69cc05590b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:11.138526 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706e7de-4511-477a-82f1-a69cc05590b3 None admin] PATH_INFO: `/v3/groups/59d498c7d42543ad9a0d6568a293c893` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:11.139768 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8706e7de-4511-477a-82f1-a69cc05590b3 None admin] RBAC: Authorizing `identity:get_group(group_id=59d498c7d42543ad9a0d6568a293c893)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:11.141272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8706e7de-4511-477a-82f1-a69cc05590b3 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:11.147990 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 192/371] 10.222.0.78 () {56 vars in 1180 bytes} [Wed May 25 14:41:10 2022] GET /identity/v3/groups/59d498c7d42543ad9a0d6568a293c893 => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:11.147990 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706e7de-4511-477a-82f1-a69cc05590b3 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:11.215586 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262b72e-10e6-47b8-b7bb-29c77fcaa7df None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:11.268219 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6d220-afc5-42d8-8eaf-81d23e9c63ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:11.269115 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6d220-afc5-42d8-8eaf-81d23e9c63ff None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:11.269376 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6d220-afc5-42d8-8eaf-81d23e9c63ff None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:11.269653 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6d220-afc5-42d8-8eaf-81d23e9c63ff None admin] PATH_INFO: `/v3/groups/59d498c7d42543ad9a0d6568a293c893` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:11.270906 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6d220-afc5-42d8-8eaf-81d23e9c63ff None admin] RBAC: Authorizing `identity:get_group(group_id=59d498c7d42543ad9a0d6568a293c893)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:11.271516 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706e7de-4511-477a-82f1-a69cc05590b3 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:11.272399 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6d220-afc5-42d8-8eaf-81d23e9c63ff None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:11.274315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 180/372] 10.222.0.78 () {56 vars in 1180 bytes} [Wed May 25 14:41:11 2022] GET /identity/v3/groups/59d498c7d42543ad9a0d6568a293c893 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:11.279781 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6d220-afc5-42d8-8eaf-81d23e9c63ff None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:11.315109 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bebbd-f184-4726-87bd-cbf51f3874b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:11.315687 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bebbd-f184-4726-87bd-cbf51f3874b3 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:11.315992 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bebbd-f184-4726-87bd-cbf51f3874b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:11.316265 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bebbd-f184-4726-87bd-cbf51f3874b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:11.367549 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:11.389330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:11.405503 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6d220-afc5-42d8-8eaf-81d23e9c63ff None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:11.412754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540bebbd-f184-4726-87bd-cbf51f3874b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:11.414265 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540bebbd-f184-4726-87bd-cbf51f3874b3 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:11.450181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0f507-9800-4d95-b859-f9b7825f5e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:11.451595 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0f507-9800-4d95-b859-f9b7825f5e44 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:11.452039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0f507-9800-4d95-b859-f9b7825f5e44 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:11.452463 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0f507-9800-4d95-b859-f9b7825f5e44 None admin] PATH_INFO: `/v3/groups/59d498c7d42543ad9a0d6568a293c893` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:11.454221 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a0f507-9800-4d95-b859-f9b7825f5e44 None admin] RBAC: Authorizing `identity:get_group(group_id=59d498c7d42543ad9a0d6568a293c893)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:11.456144 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a0f507-9800-4d95-b859-f9b7825f5e44 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:11.458150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 181/373] 10.222.0.78 () {56 vars in 1180 bytes} [Wed May 25 14:41:11 2022] GET /identity/v3/groups/59d498c7d42543ad9a0d6568a293c893 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:11.462633 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0f507-9800-4d95-b859-f9b7825f5e44 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:11.547458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 193/374] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:41:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:41:11.549111 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bebbd-f184-4726-87bd-cbf51f3874b3 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:11.589840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0f507-9800-4d95-b859-f9b7825f5e44 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:11.632499 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d101b1-ecc2-46f1-8c3e-855bec521bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:11.633575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d101b1-ecc2-46f1-8c3e-855bec521bbf None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:11.634056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d101b1-ecc2-46f1-8c3e-855bec521bbf None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:11.634506 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d101b1-ecc2-46f1-8c3e-855bec521bbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:11.635928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d101b1-ecc2-46f1-8c3e-855bec521bbf None admin] RBAC: Adding query filter params (group.id=59d498c7d42543ad9a0d6568a293c893, role.id=d451467ccbbf4149abbfc9b0f76b9339, scope.project.id=9799d178c2a042baa9963766f953c6a6) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:11.636458 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d101b1-ecc2-46f1-8c3e-855bec521bbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:11.639338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d101b1-ecc2-46f1-8c3e-855bec521bbf None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:11.654063 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 182/375] 10.222.0.78 () {56 vars in 1376 bytes} [Wed May 25 14:41:11 2022] GET /identity/v3/role_assignments?group.id=59d498c7d42543ad9a0d6568a293c893&scope.project.id=9799d178c2a042baa9963766f953c6a6&role.id=d451467ccbbf4149abbfc9b0f76b9339 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:11.654063 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d101b1-ecc2-46f1-8c3e-855bec521bbf None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:11.660250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:11.697460 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:11.727067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bebbd-f184-4726-87bd-cbf51f3874b3 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:11.776421 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71549ddd-2591-432b-9ec1-a849cc89fdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:11.777497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71549ddd-2591-432b-9ec1-a849cc89fdb8 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:11.778237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71549ddd-2591-432b-9ec1-a849cc89fdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:11.778753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71549ddd-2591-432b-9ec1-a849cc89fdb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:11.780332 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71549ddd-2591-432b-9ec1-a849cc89fdb8 None admin] RBAC: Adding query filter params (group.id=59d498c7d42543ad9a0d6568a293c893, role.id=d451467ccbbf4149abbfc9b0f76b9339, scope.project.id=212e33252b164b33ae38cf0925858a09) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:11.781210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71549ddd-2591-432b-9ec1-a849cc89fdb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:11.783121 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71549ddd-2591-432b-9ec1-a849cc89fdb8 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:11.791618 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 194/376] 10.222.0.78 () {56 vars in 1376 bytes} [Wed May 25 14:41:11 2022] GET /identity/v3/role_assignments?group.id=59d498c7d42543ad9a0d6568a293c893&scope.project.id=212e33252b164b33ae38cf0925858a09&role.id=d451467ccbbf4149abbfc9b0f76b9339 => generated 260 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:11.793148 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71549ddd-2591-432b-9ec1-a849cc89fdb8 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:11.899777 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d101b1-ecc2-46f1-8c3e-855bec521bbf None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:11.950928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ba77b-c4ae-4ff6-b3c1-dffc2978ca8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:11.951612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ba77b-c4ae-4ff6-b3c1-dffc2978ca8e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:11.951897 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ba77b-c4ae-4ff6-b3c1-dffc2978ca8e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:11.952174 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ba77b-c4ae-4ff6-b3c1-dffc2978ca8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:11.953032 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ba77b-c4ae-4ff6-b3c1-dffc2978ca8e None admin] RBAC: Adding query filter params (group.id=59d498c7d42543ad9a0d6568a293c893, role.id=a3da3c62738d4790b9dac512b1568b52, scope.project.id=212e33252b164b33ae38cf0925858a09) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:11.953349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ba77b-c4ae-4ff6-b3c1-dffc2978ca8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:11.955432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ba77b-c4ae-4ff6-b3c1-dffc2978ca8e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:11.979425 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 183/377] 10.222.0.78 () {56 vars in 1376 bytes} [Wed May 25 14:41:11 2022] GET /identity/v3/role_assignments?group.id=59d498c7d42543ad9a0d6568a293c893&scope.project.id=212e33252b164b33ae38cf0925858a09&role.id=a3da3c62738d4790b9dac512b1568b52 => generated 260 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:12.056719 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71549ddd-2591-432b-9ec1-a849cc89fdb8 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:12.116335 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f3e68-7b9b-47c2-9387-2c6b724d5155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:12.117301 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f3e68-7b9b-47c2-9387-2c6b724d5155 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:12.117572 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f3e68-7b9b-47c2-9387-2c6b724d5155 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:12.117899 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f3e68-7b9b-47c2-9387-2c6b724d5155 None admin] PATH_INFO: `/v3/projects/86b029bd8d8948dfab182a990de56b86` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:12.119455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f3e68-7b9b-47c2-9387-2c6b724d5155 None admin] RBAC: Authorizing `identity:get_project(project_id=86b029bd8d8948dfab182a990de56b86)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:12.120972 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f3e68-7b9b-47c2-9387-2c6b724d5155 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:12.123248 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 195/378] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:41:11 2022] GET /identity/v3/projects/86b029bd8d8948dfab182a990de56b86 => generated 344 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:12.132665 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ba77b-c4ae-4ff6-b3c1-dffc2978ca8e None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:12.233105 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:12.278790 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ba77b-c4ae-4ff6-b3c1-dffc2978ca8e None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:12.331662 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ba53c-cac8-4505-a27a-c7e6dae23d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:12.332733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ba53c-cac8-4505-a27a-c7e6dae23d4a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:12.333090 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ba53c-cac8-4505-a27a-c7e6dae23d4a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:12.333374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ba53c-cac8-4505-a27a-c7e6dae23d4a None admin] PATH_INFO: `/v3/groups/463a21f48e854817af55cb2d80c448bf` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:12.340923 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ba53c-cac8-4505-a27a-c7e6dae23d4a None admin] RBAC: Authorizing `identity:get_group(group_id=463a21f48e854817af55cb2d80c448bf)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:12.342486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ba53c-cac8-4505-a27a-c7e6dae23d4a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:12.345094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 184/379] 10.222.0.78 () {56 vars in 1180 bytes} [Wed May 25 14:41:12 2022] GET /identity/v3/groups/463a21f48e854817af55cb2d80c448bf => generated 229 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:12.355118 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f3e68-7b9b-47c2-9387-2c6b724d5155 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:12.385959 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:12.539631 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f3e68-7b9b-47c2-9387-2c6b724d5155 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:12.620393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fcdabf-bbea-4d7e-954a-50df42f90523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:12.621939 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fcdabf-bbea-4d7e-954a-50df42f90523 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:12.622640 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fcdabf-bbea-4d7e-954a-50df42f90523 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:12.623304 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fcdabf-bbea-4d7e-954a-50df42f90523 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:12.624898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fcdabf-bbea-4d7e-954a-50df42f90523 None admin] RBAC: Adding query filter params (group.id=463a21f48e854817af55cb2d80c448bf, role.id=e566b37cd92f467198b5dc40f183effc, scope.project.id=86b029bd8d8948dfab182a990de56b86) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:12.625578 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fcdabf-bbea-4d7e-954a-50df42f90523 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:12.627431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fcdabf-bbea-4d7e-954a-50df42f90523 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:12.645284 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 196/380] 10.222.0.78 () {56 vars in 1376 bytes} [Wed May 25 14:41:12 2022] GET /identity/v3/role_assignments?group.id=463a21f48e854817af55cb2d80c448bf&scope.project.id=86b029bd8d8948dfab182a990de56b86&role.id=e566b37cd92f467198b5dc40f183effc => generated 260 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:13.086457 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2bb61-5c1e-49ec-9639-dacdb1e27275 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:13.087229 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2bb61-5c1e-49ec-9639-dacdb1e27275 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:13.087229 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2bb61-5c1e-49ec-9639-dacdb1e27275 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:13.088507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 185/381] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:13.096246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8afccde-a335-4365-b617-a11afb3a330d None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:13.097301 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8afccde-a335-4365-b617-a11afb3a330d None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:13.097981 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8afccde-a335-4365-b617-a11afb3a330d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:13.143690 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-e8afccde-a335-4365-b617-a11afb3a330d None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:13.145962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-e8afccde-a335-4365-b617-a11afb3a330d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:13.284891 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 197/382] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:13.312448 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2bb61-5c1e-49ec-9639-dacdb1e27275 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:13.458830 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2bb61-5c1e-49ec-9639-dacdb1e27275 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:13.499444 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:13.505620 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62b563-66cf-4414-801e-47ce34f78c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:13.507083 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62b563-66cf-4414-801e-47ce34f78c05 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:13.507370 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62b563-66cf-4414-801e-47ce34f78c05 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:13.507646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62b563-66cf-4414-801e-47ce34f78c05 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:13.523045 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62b563-66cf-4414-801e-47ce34f78c05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:13.531081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-8c62b563-66cf-4414-801e-47ce34f78c05 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:41:13.531081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:13.531081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:13.531081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:13.531081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:13.531081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:13.531081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:13.531081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:13.531081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:13.531081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:13.531081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:13.531081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:13.531081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:13.531081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:13.531081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:13.531081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:13.532674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:13.532674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:13.532674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:13.532674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:13.532674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:13.532674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:13.532674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:13.532674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:13.532674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:13.532674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:13.532674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:13.532674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:13.532674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:13.532674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:13.532674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:13.532674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:13.533929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:13.533929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:13.533929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:13.533929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:13.533929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:13.533929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:13.533929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:13.533929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:13.533929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:13.533929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:13.533929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:41:13.533929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:13.533929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 186/383] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:41:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:13.540194 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8afccde-a335-4365-b617-a11afb3a330d None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:13.567890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:13.756125 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8afccde-a335-4365-b617-a11afb3a330d None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:13.813053 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92faf89-14eb-4e72-ac40-0aa66ded28f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:13.813681 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92faf89-14eb-4e72-ac40-0aa66ded28f7 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:13.813963 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92faf89-14eb-4e72-ac40-0aa66ded28f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:13.814265 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92faf89-14eb-4e72-ac40-0aa66ded28f7 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:13.815058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92faf89-14eb-4e72-ac40-0aa66ded28f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:13.815367 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92faf89-14eb-4e72-ac40-0aa66ded28f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:13.817010 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92faf89-14eb-4e72-ac40-0aa66ded28f7 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:13.831205 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 198/384] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:41:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:13.837222 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62b563-66cf-4414-801e-47ce34f78c05 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:14.024469 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62b563-66cf-4414-801e-47ce34f78c05 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:14.071935 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d012f18-5e1c-427b-a338-36cdb2b243b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:14.072788 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d012f18-5e1c-427b-a338-36cdb2b243b5 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:14.073080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d012f18-5e1c-427b-a338-36cdb2b243b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:14.073363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d012f18-5e1c-427b-a338-36cdb2b243b5 None admin] PATH_INFO: `/v3/groups/59d498c7d42543ad9a0d6568a293c893` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:14.075110 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d012f18-5e1c-427b-a338-36cdb2b243b5 None admin] RBAC: Authorizing `identity:get_group(group_id=59d498c7d42543ad9a0d6568a293c893)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:14.076623 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d012f18-5e1c-427b-a338-36cdb2b243b5 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:14.078376 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 187/385] 10.222.0.78 () {56 vars in 1180 bytes} [Wed May 25 14:41:13 2022] GET /identity/v3/groups/59d498c7d42543ad9a0d6568a293c893 => generated 226 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:14.084674 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92faf89-14eb-4e72-ac40-0aa66ded28f7 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:14.187515 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:14.233841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92faf89-14eb-4e72-ac40-0aa66ded28f7 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:14.295023 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39360186-d2b8-442e-8ab6-cf0436ec97a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:14.296310 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39360186-d2b8-442e-8ab6-cf0436ec97a0 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:14.296831 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39360186-d2b8-442e-8ab6-cf0436ec97a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:14.297846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39360186-d2b8-442e-8ab6-cf0436ec97a0 None admin] PATH_INFO: `/v3/projects/9799d178c2a042baa9963766f953c6a6` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:14.300071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39360186-d2b8-442e-8ab6-cf0436ec97a0 None admin] RBAC: Authorizing `identity:get_project(project_id=9799d178c2a042baa9963766f953c6a6)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:14.302163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39360186-d2b8-442e-8ab6-cf0436ec97a0 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:14.314190 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 199/386] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:41:14 2022] GET /identity/v3/projects/9799d178c2a042baa9963766f953c6a6 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:14.317360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d012f18-5e1c-427b-a338-36cdb2b243b5 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:14.416702 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:14.477533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d012f18-5e1c-427b-a338-36cdb2b243b5 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:14.482727 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b0c1b-fce7-4171-9ada-649035c80c21 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:14.483450 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b0c1b-fce7-4171-9ada-649035c80c21 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:14.483924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b0c1b-fce7-4171-9ada-649035c80c21 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:14.485983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 200/387] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:14.498222 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515791b6-971c-4b7a-b157-e012eb54a6d9 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:14.498835 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515791b6-971c-4b7a-b157-e012eb54a6d9 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:14.499314 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515791b6-971c-4b7a-b157-e012eb54a6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:14.519888 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-515791b6-971c-4b7a-b157-e012eb54a6d9 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:14.521478 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-515791b6-971c-4b7a-b157-e012eb54a6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:14.601773 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 201/388] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:41:14.603243 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412c7cb-2568-4ff9-ae1f-b3b8c2953339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:14.604783 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412c7cb-2568-4ff9-ae1f-b3b8c2953339 None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:14.605298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412c7cb-2568-4ff9-ae1f-b3b8c2953339 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:14.605762 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412c7cb-2568-4ff9-ae1f-b3b8c2953339 None admin] PATH_INFO: `/v3/projects/9799d178c2a042baa9963766f953c6a6/groups/59d498c7d42543ad9a0d6568a293c893/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:14.609621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9412c7cb-2568-4ff9-ae1f-b3b8c2953339 None admin] RBAC: Authorizing `identity:create_grant(project_id=9799d178c2a042baa9963766f953c6a6, group_id=59d498c7d42543ad9a0d6568a293c893, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:14.612921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9412c7cb-2568-4ff9-ae1f-b3b8c2953339 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:14.642132 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 188/389] 10.222.0.78 () {58 vars in 1442 bytes} [Wed May 25 14:41:14 2022] PUT /identity/v3/projects/9799d178c2a042baa9963766f953c6a6/groups/59d498c7d42543ad9a0d6568a293c893/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:14.653026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515791b6-971c-4b7a-b157-e012eb54a6d9 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:14.738720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:14.790994 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515791b6-971c-4b7a-b157-e012eb54a6d9 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:14.856687 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3a83e-009b-4076-82c0-fa40c8c930cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:14.857930 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3a83e-009b-4076-82c0-fa40c8c930cf None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:14.858448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3a83e-009b-4076-82c0-fa40c8c930cf None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:14.859027 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3a83e-009b-4076-82c0-fa40c8c930cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:14.882944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3a83e-009b-4076-82c0-fa40c8c930cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:14.892295 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-0fd3a83e-009b-4076-82c0-fa40c8c930cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:41:14.892295 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:14.892295 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:14.892295 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:14.892295 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:14.892295 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:14.892295 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:14.892295 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:14.892295 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:14.892295 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:14.892295 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:14.892295 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:14.892295 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:14.892295 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:14.892295 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:14.892295 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:14.893302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:14.893302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:14.893302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:14.893302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:14.893302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:14.893302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:14.893302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:14.893302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:14.893302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:14.893302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:14.893302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:14.893302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:14.893302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:14.893302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:14.893302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:14.893302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:14.894126 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:14.894126 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:14.894126 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:14.894126 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:14.894126 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:14.894126 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:14.894126 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:14.894126 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:14.894126 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:14.894126 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:14.894126 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:41:14.894126 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:14.896733 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 202/390] 10.222.0.78 () {56 vars in 1114 bytes} [Wed May 25 14:41:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:14.927949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412c7cb-2568-4ff9-ae1f-b3b8c2953339 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:15.144669 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412c7cb-2568-4ff9-ae1f-b3b8c2953339 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:15.190265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:15.225562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1b36a-8ff3-423a-9356-a49211cc1c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:15.226799 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1b36a-8ff3-423a-9356-a49211cc1c6d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:15.227273 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1b36a-8ff3-423a-9356-a49211cc1c6d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:15.227773 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1b36a-8ff3-423a-9356-a49211cc1c6d None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:15.229292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1b36a-8ff3-423a-9356-a49211cc1c6d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:15.229863 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1b36a-8ff3-423a-9356-a49211cc1c6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:15.232706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1b36a-8ff3-423a-9356-a49211cc1c6d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:15.256298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 189/391] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:41:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:15.271202 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3a83e-009b-4076-82c0-fa40c8c930cf None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:15.306266 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1617c48-a4b5-4893-b677-708c387e45bb None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:15.307101 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1617c48-a4b5-4893-b677-708c387e45bb None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:15.307792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1617c48-a4b5-4893-b677-708c387e45bb None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:15.310714 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 190/392] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:15.323508 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61737a83-2605-4712-81ec-353aba97d36c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:15.325031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61737a83-2605-4712-81ec-353aba97d36c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:15.325720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61737a83-2605-4712-81ec-353aba97d36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:15.362910 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-61737a83-2605-4712-81ec-353aba97d36c None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:15.364209 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-61737a83-2605-4712-81ec-353aba97d36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:15.475227 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3a83e-009b-4076-82c0-fa40c8c930cf None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:15.506182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:15.516481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a207e82-fc36-445b-a549-b77db734c5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:15.517708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a207e82-fc36-445b-a549-b77db734c5e0 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:15.518484 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a207e82-fc36-445b-a549-b77db734c5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:15.519035 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 191/393] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:15.519178 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a207e82-fc36-445b-a549-b77db734c5e0 None admin] PATH_INFO: `/v3/groups/59d498c7d42543ad9a0d6568a293c893` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:15.520420 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34125846-dc7f-40ed-bfe3-63908f6d3857 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:15.526076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34125846-dc7f-40ed-bfe3-63908f6d3857 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:15.526076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34125846-dc7f-40ed-bfe3-63908f6d3857 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:15.526076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 192/394] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:15.531030 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a207e82-fc36-445b-a549-b77db734c5e0 None admin] RBAC: Authorizing `identity:get_group(group_id=59d498c7d42543ad9a0d6568a293c893)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:15.532620 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a207e82-fc36-445b-a549-b77db734c5e0 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:15.533360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ce2ac-4df7-4ae8-be0d-5447e7957d21 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:15.534319 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ce2ac-4df7-4ae8-be0d-5447e7957d21 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:15.534549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 203/395] 10.222.0.78 () {56 vars in 1180 bytes} [Wed May 25 14:41:15 2022] GET /identity/v3/groups/59d498c7d42543ad9a0d6568a293c893 => generated 226 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:15.535031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ce2ac-4df7-4ae8-be0d-5447e7957d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:15.543829 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a207e82-fc36-445b-a549-b77db734c5e0 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:15.566684 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-071ce2ac-4df7-4ae8-be0d-5447e7957d21 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:15.568775 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-071ce2ac-4df7-4ae8-be0d-5447e7957d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:15.683485 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 193/396] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:15.684724 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ce2ac-4df7-4ae8-be0d-5447e7957d21 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:15.739614 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:15.748392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a207e82-fc36-445b-a549-b77db734c5e0 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:15.787214 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10d015-c382-4e92-ae4b-ed8697bb1e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:15.788793 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10d015-c382-4e92-ae4b-ed8697bb1e0a None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:15.789251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10d015-c382-4e92-ae4b-ed8697bb1e0a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:15.789697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10d015-c382-4e92-ae4b-ed8697bb1e0a None admin] PATH_INFO: `/v3/projects/9799d178c2a042baa9963766f953c6a6` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:15.791321 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10d015-c382-4e92-ae4b-ed8697bb1e0a None admin] RBAC: Authorizing `identity:get_project(project_id=9799d178c2a042baa9963766f953c6a6)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:15.793013 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10d015-c382-4e92-ae4b-ed8697bb1e0a None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:15.795732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 204/397] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:41:15 2022] GET /identity/v3/projects/9799d178c2a042baa9963766f953c6a6 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:15.796804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10d015-c382-4e92-ae4b-ed8697bb1e0a None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:15.815469 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ce2ac-4df7-4ae8-be0d-5447e7957d21 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:15.867571 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273a1e3-8c86-417c-b3c6-a8549d7e112f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:15.869588 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273a1e3-8c86-417c-b3c6-a8549d7e112f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:15.870150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273a1e3-8c86-417c-b3c6-a8549d7e112f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:15.870861 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273a1e3-8c86-417c-b3c6-a8549d7e112f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:15.888684 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e273a1e3-8c86-417c-b3c6-a8549d7e112f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:15.897830 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:15.899176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-e273a1e3-8c86-417c-b3c6-a8549d7e112f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:41:15.899176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:15.899176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:15.899176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:15.899176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:15.899176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:15.899176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:15.899176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:15.899176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:15.899176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:15.899176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:15.899176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:15.899176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:15.899176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:15.899176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:15.899176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:15.900346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:15.900346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:15.900346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:15.900346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:15.900346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:15.900346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:15.900346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:15.900346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:15.900346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:15.900346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:15.900346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:15.900346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:15.900346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:15.900346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:15.900346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:15.900346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:15.901329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:15.901329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:15.901329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:15.901329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:15.901329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:15.901329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:15.901329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:15.901329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:15.901329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:15.901329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:15.901329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:41:15.901329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:15.904110 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 194/398] 10.222.0.78 () {56 vars in 1114 bytes} [Wed May 25 14:41:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:15.905160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273a1e3-8c86-417c-b3c6-a8549d7e112f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:15.973226 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10d015-c382-4e92-ae4b-ed8697bb1e0a None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:16.015677 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa07b55-aa3e-4e49-bcef-1fa7bed1b8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:16.017590 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa07b55-aa3e-4e49-bcef-1fa7bed1b8e6 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:16.017781 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa07b55-aa3e-4e49-bcef-1fa7bed1b8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:16.018000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa07b55-aa3e-4e49-bcef-1fa7bed1b8e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:16.035103 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273a1e3-8c86-417c-b3c6-a8549d7e112f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:16.035350 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa07b55-aa3e-4e49-bcef-1fa7bed1b8e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:16.037636 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:16.050522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-0fa07b55-aa3e-4e49-bcef-1fa7bed1b8e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:41:16.050522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:16.050522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:16.050522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:16.050522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:16.050522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:16.050522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:16.050522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:16.050522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:16.050522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:16.050522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:16.050522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:16.050522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:16.050522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:16.050522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:16.050522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:16.051481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:16.051481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:16.051481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:16.051481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:16.051481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:16.051481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:16.051481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:16.051481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:16.051481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:16.051481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:16.051481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:16.051481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:16.051481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:16.051481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:16.051481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:16.051481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:16.052249 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:16.052249 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:16.052249 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:16.052249 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:16.052249 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:16.052249 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:16.052249 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:16.052249 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:16.052249 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:16.052249 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:16.052249 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:41:16.052249 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:16.053427 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 205/399] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:41:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:16.055750 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119a9c5-b7b3-46b9-8dba-397913e3bf7b None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:16.056287 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119a9c5-b7b3-46b9-8dba-397913e3bf7b None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:16.056805 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119a9c5-b7b3-46b9-8dba-397913e3bf7b None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:16.059751 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 206/400] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:16.061193 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119a9c5-b7b3-46b9-8dba-397913e3bf7b None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:16.110501 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d932933-97bf-4675-8871-43728e37930a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:16.111912 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d932933-97bf-4675-8871-43728e37930a None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:16.112373 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d932933-97bf-4675-8871-43728e37930a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:16.112841 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d932933-97bf-4675-8871-43728e37930a None admin] PATH_INFO: `/v3/projects/9799d178c2a042baa9963766f953c6a6/groups/59d498c7d42543ad9a0d6568a293c893/roles/d451467ccbbf4149abbfc9b0f76b9339` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:16.115988 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d932933-97bf-4675-8871-43728e37930a None admin] RBAC: Authorizing `identity:create_grant(project_id=9799d178c2a042baa9963766f953c6a6, group_id=59d498c7d42543ad9a0d6568a293c893, role_id=d451467ccbbf4149abbfc9b0f76b9339)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:16.118434 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d932933-97bf-4675-8871-43728e37930a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:16.141995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 195/401] 10.222.0.78 () {58 vars in 1442 bytes} [Wed May 25 14:41:15 2022] PUT /identity/v3/projects/9799d178c2a042baa9963766f953c6a6/groups/59d498c7d42543ad9a0d6568a293c893/roles/d451467ccbbf4149abbfc9b0f76b9339 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:16.143986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d932933-97bf-4675-8871-43728e37930a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:16.272584 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119a9c5-b7b3-46b9-8dba-397913e3bf7b None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:16.323520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:16.346230 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579e215-2089-4eda-8caf-281978fbf33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:16.347770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579e215-2089-4eda-8caf-281978fbf33a None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:16.348554 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579e215-2089-4eda-8caf-281978fbf33a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:16.349284 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579e215-2089-4eda-8caf-281978fbf33a None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:16.351001 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579e215-2089-4eda-8caf-281978fbf33a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:16.351887 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579e215-2089-4eda-8caf-281978fbf33a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:16.354938 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579e215-2089-4eda-8caf-281978fbf33a None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:16.356980 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d932933-97bf-4675-8871-43728e37930a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:16.373296 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 207/402] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:41:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:16.376675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3af0ec-f4bd-497a-91f0-e18c1defdd39 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:16.377537 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3af0ec-f4bd-497a-91f0-e18c1defdd39 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:16.380607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3af0ec-f4bd-497a-91f0-e18c1defdd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:16.398787 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-fe3af0ec-f4bd-497a-91f0-e18c1defdd39 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:16.400751 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-fe3af0ec-f4bd-497a-91f0-e18c1defdd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:16.408514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502207a-cb7d-433a-af2f-2567b4a872be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:16.409528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502207a-cb7d-433a-af2f-2567b4a872be None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:16.409978 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502207a-cb7d-433a-af2f-2567b4a872be None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:16.410518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502207a-cb7d-433a-af2f-2567b4a872be None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:16.412880 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9502207a-cb7d-433a-af2f-2567b4a872be None admin] RBAC: Adding query filter params (name=member) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:16.412880 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9502207a-cb7d-433a-af2f-2567b4a872be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:16.415878 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9502207a-cb7d-433a-af2f-2567b4a872be None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:16.428456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 196/403] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:41:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:16.429809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502207a-cb7d-433a-af2f-2567b4a872be None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:16.467850 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:16.542107 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 208/404] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:16.543923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3af0ec-f4bd-497a-91f0-e18c1defdd39 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:16.586065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502207a-cb7d-433a-af2f-2567b4a872be None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:16.635977 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ad6be-eaef-4a94-bc5c-7bba781e2090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:16.636866 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ad6be-eaef-4a94-bc5c-7bba781e2090 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:16.637152 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ad6be-eaef-4a94-bc5c-7bba781e2090 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:16.637432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ad6be-eaef-4a94-bc5c-7bba781e2090 None admin] PATH_INFO: `/v3/groups/59d498c7d42543ad9a0d6568a293c893` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:16.638948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ad6be-eaef-4a94-bc5c-7bba781e2090 None admin] RBAC: Authorizing `identity:get_group(group_id=59d498c7d42543ad9a0d6568a293c893)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:16.640846 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ad6be-eaef-4a94-bc5c-7bba781e2090 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:16.648419 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 197/405] 10.222.0.78 () {56 vars in 1180 bytes} [Wed May 25 14:41:16 2022] GET /identity/v3/groups/59d498c7d42543ad9a0d6568a293c893 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:16.649435 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ad6be-eaef-4a94-bc5c-7bba781e2090 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:16.737396 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3af0ec-f4bd-497a-91f0-e18c1defdd39 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:16.770985 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:16.778418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:16.783591 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ad6be-eaef-4a94-bc5c-7bba781e2090 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:16.810703 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec52f1-0389-4544-85c2-def71985d197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:16.812634 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec52f1-0389-4544-85c2-def71985d197 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:16.814092 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec52f1-0389-4544-85c2-def71985d197 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:16.815010 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec52f1-0389-4544-85c2-def71985d197 None admin] PATH_INFO: `/v3/groups/59d498c7d42543ad9a0d6568a293c893` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:16.817787 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ec52f1-0389-4544-85c2-def71985d197 None admin] RBAC: Authorizing `identity:get_group(group_id=59d498c7d42543ad9a0d6568a293c893)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:16.821119 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ec52f1-0389-4544-85c2-def71985d197 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:16.824858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 209/406] 10.222.0.78 () {56 vars in 1180 bytes} [Wed May 25 14:41:16 2022] GET /identity/v3/groups/59d498c7d42543ad9a0d6568a293c893 => generated 226 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:16.826755 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec52f1-0389-4544-85c2-def71985d197 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:16.830610 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41c300-6506-404f-bcb4-b640905fa0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:16.831429 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41c300-6506-404f-bcb4-b640905fa0be None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:16.831742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41c300-6506-404f-bcb4-b640905fa0be None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:16.831992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41c300-6506-404f-bcb4-b640905fa0be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:16.851497 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb41c300-6506-404f-bcb4-b640905fa0be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:16.861582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-fb41c300-6506-404f-bcb4-b640905fa0be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:41:16.861582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:16.861582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:16.861582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:16.861582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:16.861582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:16.861582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:16.861582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:16.861582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:16.861582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:16.861582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:16.861582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:16.861582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:16.861582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:16.861582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:16.861582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:16.862584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:16.862584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:16.862584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:16.862584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:16.862584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:16.862584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:16.862584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:16.862584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:16.862584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:16.862584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:16.862584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:16.862584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:16.862584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:16.862584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:16.862584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:16.862584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:16.863361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:16.863361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:16.863361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:16.863361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:16.863361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:16.863361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:16.863361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:16.863361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:16.863361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:16.863361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:16.863361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:41:16.863361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:16.866331 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 198/407] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:41:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:16.867400 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41c300-6506-404f-bcb4-b640905fa0be None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:17.010252 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41c300-6506-404f-bcb4-b640905fa0be None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:17.055416 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec52f1-0389-4544-85c2-def71985d197 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:17.078073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c655a93-b156-4a10-a131-bb3a42e8264c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:17.080146 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c655a93-b156-4a10-a131-bb3a42e8264c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:17.080469 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c655a93-b156-4a10-a131-bb3a42e8264c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:17.080751 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c655a93-b156-4a10-a131-bb3a42e8264c None admin] PATH_INFO: `/v3/projects/212e33252b164b33ae38cf0925858a09` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:17.082302 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c655a93-b156-4a10-a131-bb3a42e8264c None admin] RBAC: Authorizing `identity:get_project(project_id=212e33252b164b33ae38cf0925858a09)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:17.084192 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c655a93-b156-4a10-a131-bb3a42e8264c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:17.086695 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 199/408] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:41:16 2022] GET /identity/v3/projects/212e33252b164b33ae38cf0925858a09 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:17.087576 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c655a93-b156-4a10-a131-bb3a42e8264c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:17.120420 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:17.131725 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843323fd-9e46-4efa-93c7-e6ed6ef40f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:17.134336 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843323fd-9e46-4efa-93c7-e6ed6ef40f89 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:17.135289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843323fd-9e46-4efa-93c7-e6ed6ef40f89 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:17.136116 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843323fd-9e46-4efa-93c7-e6ed6ef40f89 None admin] PATH_INFO: `/v3/projects/212e33252b164b33ae38cf0925858a09` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:17.139324 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843323fd-9e46-4efa-93c7-e6ed6ef40f89 None admin] RBAC: Authorizing `identity:get_project(project_id=212e33252b164b33ae38cf0925858a09)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:17.143475 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843323fd-9e46-4efa-93c7-e6ed6ef40f89 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:17.148265 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 210/409] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:41:16 2022] GET /identity/v3/projects/212e33252b164b33ae38cf0925858a09 => generated 302 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:17.150413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843323fd-9e46-4efa-93c7-e6ed6ef40f89 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:17.229657 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c655a93-b156-4a10-a131-bb3a42e8264c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:17.260480 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:17.289786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e959d-b271-4e5a-ab5f-271ea6bd3748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:17.291459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e959d-b271-4e5a-ab5f-271ea6bd3748 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:17.292199 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e959d-b271-4e5a-ab5f-271ea6bd3748 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:17.293008 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e959d-b271-4e5a-ab5f-271ea6bd3748 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:17.294779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e959d-b271-4e5a-ab5f-271ea6bd3748 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:17.295610 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e959d-b271-4e5a-ab5f-271ea6bd3748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:17.299328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e959d-b271-4e5a-ab5f-271ea6bd3748 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:17.322543 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 200/410] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:41:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:17.322900 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843323fd-9e46-4efa-93c7-e6ed6ef40f89 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:17.325232 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e959d-b271-4e5a-ab5f-271ea6bd3748 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:17.368457 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f788e4-68ae-4e70-b0dc-94b72b4618da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:17.369912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f788e4-68ae-4e70-b0dc-94b72b4618da None admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:17.370227 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f788e4-68ae-4e70-b0dc-94b72b4618da None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:17.370663 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f788e4-68ae-4e70-b0dc-94b72b4618da None admin] PATH_INFO: `/v3/projects/212e33252b164b33ae38cf0925858a09/groups/59d498c7d42543ad9a0d6568a293c893/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:17.373329 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f788e4-68ae-4e70-b0dc-94b72b4618da None admin] RBAC: Authorizing `identity:create_grant(project_id=212e33252b164b33ae38cf0925858a09, group_id=59d498c7d42543ad9a0d6568a293c893, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:17.375763 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f788e4-68ae-4e70-b0dc-94b72b4618da None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:17.395112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 211/411] 10.222.0.78 () {58 vars in 1442 bytes} [Wed May 25 14:41:17 2022] PUT /identity/v3/projects/212e33252b164b33ae38cf0925858a09/groups/59d498c7d42543ad9a0d6568a293c893/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:17.396258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f788e4-68ae-4e70-b0dc-94b72b4618da None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:17.488926 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:17.553708 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e959d-b271-4e5a-ab5f-271ea6bd3748 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:17.664161 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f788e4-68ae-4e70-b0dc-94b72b4618da None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:17.694338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbdcdf-62f5-4974-a6d7-0f94461d70dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:17.696480 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbdcdf-62f5-4974-a6d7-0f94461d70dc None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:17.697740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbdcdf-62f5-4974-a6d7-0f94461d70dc None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:17.700318 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbdcdf-62f5-4974-a6d7-0f94461d70dc None admin] PATH_INFO: `/v3/projects/212e33252b164b33ae38cf0925858a09/groups/59d498c7d42543ad9a0d6568a293c893/roles/d451467ccbbf4149abbfc9b0f76b9339` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:17.707652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedbdcdf-62f5-4974-a6d7-0f94461d70dc None admin] RBAC: Authorizing `identity:create_grant(project_id=212e33252b164b33ae38cf0925858a09, group_id=59d498c7d42543ad9a0d6568a293c893, role_id=d451467ccbbf4149abbfc9b0f76b9339)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:17.710638 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:17.713111 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedbdcdf-62f5-4974-a6d7-0f94461d70dc None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:17.740980 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b5144b-82d9-4802-a78d-27b5dfc3d856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:17.742405 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b5144b-82d9-4802-a78d-27b5dfc3d856 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:17.742983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b5144b-82d9-4802-a78d-27b5dfc3d856 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:17.743484 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b5144b-82d9-4802-a78d-27b5dfc3d856 None admin] PATH_INFO: `/v3/groups/463a21f48e854817af55cb2d80c448bf` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:17.745025 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b5144b-82d9-4802-a78d-27b5dfc3d856 None admin] RBAC: Authorizing `identity:get_group(group_id=463a21f48e854817af55cb2d80c448bf)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:17.747092 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b5144b-82d9-4802-a78d-27b5dfc3d856 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:17.749416 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 212/412] 10.222.0.78 () {56 vars in 1180 bytes} [Wed May 25 14:41:17 2022] GET /identity/v3/groups/463a21f48e854817af55cb2d80c448bf => generated 229 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:17.753026 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 201/413] 10.222.0.78 () {58 vars in 1442 bytes} [Wed May 25 14:41:17 2022] PUT /identity/v3/projects/212e33252b164b33ae38cf0925858a09/groups/59d498c7d42543ad9a0d6568a293c893/roles/d451467ccbbf4149abbfc9b0f76b9339 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:17.757532 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b5144b-82d9-4802-a78d-27b5dfc3d856 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:17.882165 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b5144b-82d9-4802-a78d-27b5dfc3d856 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:17.924683 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a4d51-0557-4a20-ba27-c87b9e78c760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:17.925632 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a4d51-0557-4a20-ba27-c87b9e78c760 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:17.926028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a4d51-0557-4a20-ba27-c87b9e78c760 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:17.926396 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a4d51-0557-4a20-ba27-c87b9e78c760 None admin] PATH_INFO: `/v3/projects/86b029bd8d8948dfab182a990de56b86` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:17.927803 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a4d51-0557-4a20-ba27-c87b9e78c760 None admin] RBAC: Authorizing `identity:get_project(project_id=86b029bd8d8948dfab182a990de56b86)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:17.929397 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a4d51-0557-4a20-ba27-c87b9e78c760 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:17.931683 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 213/414] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:41:17 2022] GET /identity/v3/projects/86b029bd8d8948dfab182a990de56b86 => generated 344 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:17.941012 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbdcdf-62f5-4974-a6d7-0f94461d70dc None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:18.187403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbdcdf-62f5-4974-a6d7-0f94461d70dc None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:18.251219 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2534c40f-59e4-4c4f-843a-579b3ad62c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:18.254402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534c40f-59e4-4c4f-843a-579b3ad62c39 None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:18.254682 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534c40f-59e4-4c4f-843a-579b3ad62c39 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:18.254960 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534c40f-59e4-4c4f-843a-579b3ad62c39 None admin] PATH_INFO: `/v3/projects/86b029bd8d8948dfab182a990de56b86/groups/463a21f48e854817af55cb2d80c448bf/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:18.257418 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2534c40f-59e4-4c4f-843a-579b3ad62c39 None admin] RBAC: Authorizing `identity:create_grant(project_id=86b029bd8d8948dfab182a990de56b86, group_id=463a21f48e854817af55cb2d80c448bf, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:18.259014 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2534c40f-59e4-4c4f-843a-579b3ad62c39 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:18.290562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 202/415] 10.222.0.78 () {58 vars in 1442 bytes} [Wed May 25 14:41:17 2022] PUT /identity/v3/projects/86b029bd8d8948dfab182a990de56b86/groups/463a21f48e854817af55cb2d80c448bf/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:18.492913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e03d8-2d55-4074-b737-46ccc060db18 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:18.493476 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e03d8-2d55-4074-b737-46ccc060db18 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:18.493476 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e03d8-2d55-4074-b737-46ccc060db18 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:18.495119 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 214/416] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:18.507918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f289a9-56af-4293-8170-23ee083ae58e None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:18.508436 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f289a9-56af-4293-8170-23ee083ae58e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:18.508785 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f289a9-56af-4293-8170-23ee083ae58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:18.520685 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:18.543481 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-76f289a9-56af-4293-8170-23ee083ae58e None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:18.545465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-76f289a9-56af-4293-8170-23ee083ae58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:18.699393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 203/417] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:18.731348 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e03d8-2d55-4074-b737-46ccc060db18 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:18.797317 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:18.855291 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e03d8-2d55-4074-b737-46ccc060db18 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:18.893150 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb1b41-6c8c-4620-b7ab-11b502c3dc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:18.893955 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb1b41-6c8c-4620-b7ab-11b502c3dc86 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:18.894241 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb1b41-6c8c-4620-b7ab-11b502c3dc86 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:18.894517 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb1b41-6c8c-4620-b7ab-11b502c3dc86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:18.908139 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe21cfb-8b02-4586-bfb6-0ed001cef5c4 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:18.908139 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe21cfb-8b02-4586-bfb6-0ed001cef5c4 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:18.908139 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe21cfb-8b02-4586-bfb6-0ed001cef5c4 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:18.908753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb1b41-6c8c-4620-b7ab-11b502c3dc86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:18.908893 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 204/418] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:18.918799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-5adb1b41-6c8c-4620-b7ab-11b502c3dc86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:41:18.918799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:18.918799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:18.918799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:18.918799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:18.918799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:18.918799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:18.918799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:18.918799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:18.918799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:18.918799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:18.918799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:18.918799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:18.918799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:18.918799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:18.918799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:18.920660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:18.920660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:18.920660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:18.920660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:18.920660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:18.920660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:18.920660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:18.920660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:18.920660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:18.920660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:18.920660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:18.920660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:18.920660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:18.920660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:18.920660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:18.920660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:18.921670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:18.921670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:18.921670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:18.921670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:18.921670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:18.921670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:18.921670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:18.921670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:18.921670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:18.921670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:18.921670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:41:18.921670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:18.924808 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f20bb0-8c97-4863-9aca-d070627973df None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:18.924808 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f20bb0-8c97-4863-9aca-d070627973df None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:18.925095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 215/419] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:41:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:18.925254 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f20bb0-8c97-4863-9aca-d070627973df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:18.925459 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb1b41-6c8c-4620-b7ab-11b502c3dc86 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:18.975243 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-52f20bb0-8c97-4863-9aca-d070627973df None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:18.976469 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-52f20bb0-8c97-4863-9aca-d070627973df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:19.044969 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb1b41-6c8c-4620-b7ab-11b502c3dc86 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:19.058509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:19.088785 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059aca6-fe62-4e79-a663-9f7175c23f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:19.089464 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059aca6-fe62-4e79-a663-9f7175c23f7b None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:19.089751 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059aca6-fe62-4e79-a663-9f7175c23f7b None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:19.090005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059aca6-fe62-4e79-a663-9f7175c23f7b None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:19.090779 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059aca6-fe62-4e79-a663-9f7175c23f7b None admin] RBAC: Adding query filter params (name=member) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:19.091083 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059aca6-fe62-4e79-a663-9f7175c23f7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:19.092481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059aca6-fe62-4e79-a663-9f7175c23f7b None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:19.122527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 216/420] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:41:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:19.130536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059aca6-fe62-4e79-a663-9f7175c23f7b None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:19.205187 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 205/421] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:19.231813 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f20bb0-8c97-4863-9aca-d070627973df None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:19.271745 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059aca6-fe62-4e79-a663-9f7175c23f7b None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:19.312162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87106ded-031a-4188-8e64-0990f32a6fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:19.312922 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87106ded-031a-4188-8e64-0990f32a6fbb None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:19.313207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87106ded-031a-4188-8e64-0990f32a6fbb None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:19.313558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87106ded-031a-4188-8e64-0990f32a6fbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:19.365826 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:19.397794 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87106ded-031a-4188-8e64-0990f32a6fbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:19.399329 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87106ded-031a-4188-8e64-0990f32a6fbb None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:19.442556 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:19.522187 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f20bb0-8c97-4863-9aca-d070627973df None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:19.527463 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 217/422] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:41:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:41:19.533993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87106ded-031a-4188-8e64-0990f32a6fbb None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:19.612587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f95b2a-0a38-4e9b-ae62-b8061d98adb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:19.613435 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f95b2a-0a38-4e9b-ae62-b8061d98adb7 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:19.613718 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f95b2a-0a38-4e9b-ae62-b8061d98adb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:19.613997 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f95b2a-0a38-4e9b-ae62-b8061d98adb7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:19.622589 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:19.654811 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f95b2a-0a38-4e9b-ae62-b8061d98adb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:19.660325 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87106ded-031a-4188-8e64-0990f32a6fbb None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:19.671357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-07f95b2a-0a38-4e9b-ae62-b8061d98adb7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:41:19.671357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:19.671357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:19.671357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:19.671357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:19.671357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:19.671357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:19.671357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:19.671357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:19.671357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:19.671357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:19.671357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:19.671357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:19.671357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:19.671357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:19.671357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:19.672380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:19.672380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:19.672380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:19.672380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:19.672380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:19.672380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:19.672380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:19.672380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:19.672380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:19.672380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:19.672380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:19.672380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:19.672380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:19.672380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:19.672380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:19.672380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:19.673177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:19.673177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:19.673177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:19.673177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:19.673177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:19.673177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:19.673177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:19.673177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:19.673177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:19.673177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:19.673177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:41:19.673177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:19.675984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 206/423] 10.222.0.78 () {56 vars in 1114 bytes} [Wed May 25 14:41:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 445 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:19.682093 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f95b2a-0a38-4e9b-ae62-b8061d98adb7 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:19.715303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7bcb2-191b-455d-8846-24c4748aab3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:19.716175 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7bcb2-191b-455d-8846-24c4748aab3b None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:19.716526 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7bcb2-191b-455d-8846-24c4748aab3b None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:19.716798 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7bcb2-191b-455d-8846-24c4748aab3b None admin] PATH_INFO: `/v3/projects/9799d178c2a042baa9963766f953c6a6` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:19.722205 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7bcb2-191b-455d-8846-24c4748aab3b None admin] RBAC: Authorizing `identity:get_project(project_id=9799d178c2a042baa9963766f953c6a6)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:19.723727 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7bcb2-191b-455d-8846-24c4748aab3b None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:19.725607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 218/424] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:41:19 2022] GET /identity/v3/projects/9799d178c2a042baa9963766f953c6a6 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:19.743580 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7bcb2-191b-455d-8846-24c4748aab3b None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:19.902584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f95b2a-0a38-4e9b-ae62-b8061d98adb7 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:19.930316 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:19.948120 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5eb75-a21b-48a2-a405-edd4ea2432b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:19.949074 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5eb75-a21b-48a2-a405-edd4ea2432b6 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:19.949375 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5eb75-a21b-48a2-a405-edd4ea2432b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:19.949652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5eb75-a21b-48a2-a405-edd4ea2432b6 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:19.952189 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b5eb75-a21b-48a2-a405-edd4ea2432b6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:19.952504 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b5eb75-a21b-48a2-a405-edd4ea2432b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:19.956081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b5eb75-a21b-48a2-a405-edd4ea2432b6 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:19.976818 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 207/425] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:41:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:19.979704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5eb75-a21b-48a2-a405-edd4ea2432b6 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:20.020155 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7bcb2-191b-455d-8846-24c4748aab3b None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:20.113621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5eb75-a21b-48a2-a405-edd4ea2432b6 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:20.139481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c26e3-470e-43b0-ae57-9d18fc1d1607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:20.140366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c26e3-470e-43b0-ae57-9d18fc1d1607 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:20.140648 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c26e3-470e-43b0-ae57-9d18fc1d1607 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:20.141459 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c26e3-470e-43b0-ae57-9d18fc1d1607 None admin] PATH_INFO: `/v3/groups/59d498c7d42543ad9a0d6568a293c893` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:20.142775 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6c26e3-470e-43b0-ae57-9d18fc1d1607 None admin] RBAC: Authorizing `identity:get_group(group_id=59d498c7d42543ad9a0d6568a293c893)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:20.144365 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6c26e3-470e-43b0-ae57-9d18fc1d1607 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:20.146459 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 219/426] 10.222.0.78 () {56 vars in 1180 bytes} [Wed May 25 14:41:19 2022] GET /identity/v3/groups/59d498c7d42543ad9a0d6568a293c893 => generated 226 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:20.162465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbf393-ee0c-4d2b-9f28-5f01f3e509aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:20.163109 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbf393-ee0c-4d2b-9f28-5f01f3e509aa None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:20.163319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c26e3-470e-43b0-ae57-9d18fc1d1607 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:20.163432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbf393-ee0c-4d2b-9f28-5f01f3e509aa None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:20.163648 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbf393-ee0c-4d2b-9f28-5f01f3e509aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:20.175158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:20.238705 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:20.263550 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecbf393-ee0c-4d2b-9f28-5f01f3e509aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:20.265546 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecbf393-ee0c-4d2b-9f28-5f01f3e509aa None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:20.381649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c26e3-470e-43b0-ae57-9d18fc1d1607 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:20.387430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 208/427] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:41:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:41:20.389061 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb81c7-8ff3-4a95-8f00-ecd21284d95e None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:20.389456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb81c7-8ff3-4a95-8f00-ecd21284d95e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:20.389831 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb81c7-8ff3-4a95-8f00-ecd21284d95e None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:20.391598 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 209/428] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:20.396889 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb81c7-8ff3-4a95-8f00-ecd21284d95e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:20.423075 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb54eec-86e8-45eb-8f29-3b106e8b7ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:20.423670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb54eec-86e8-45eb-8f29-3b106e8b7ab2 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:20.423947 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb54eec-86e8-45eb-8f29-3b106e8b7ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:20.424242 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb54eec-86e8-45eb-8f29-3b106e8b7ab2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:20.425106 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb54eec-86e8-45eb-8f29-3b106e8b7ab2 None admin] RBAC: Adding query filter params (group.id=59d498c7d42543ad9a0d6568a293c893, role.id=a3da3c62738d4790b9dac512b1568b52, scope.project.id=9799d178c2a042baa9963766f953c6a6) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:20.425415 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb54eec-86e8-45eb-8f29-3b106e8b7ab2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:20.427053 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb54eec-86e8-45eb-8f29-3b106e8b7ab2 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:20.434280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 220/429] 10.222.0.78 () {56 vars in 1376 bytes} [Wed May 25 14:41:20 2022] GET /identity/v3/role_assignments?group.id=59d498c7d42543ad9a0d6568a293c893&scope.project.id=9799d178c2a042baa9963766f953c6a6&role.id=a3da3c62738d4790b9dac512b1568b52 => generated 611 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:20.435430 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75589c13-e1d0-4ed1-956b-09b3a56f6867 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:20.435820 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75589c13-e1d0-4ed1-956b-09b3a56f6867 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:20.436100 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75589c13-e1d0-4ed1-956b-09b3a56f6867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:20.455297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-75589c13-e1d0-4ed1-956b-09b3a56f6867 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:20.456569 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-75589c13-e1d0-4ed1-956b-09b3a56f6867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:20.535242 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:20.540918 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 221/430] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:20.558957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:20.570058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75589c13-e1d0-4ed1-956b-09b3a56f6867 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:20.612879 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb81c7-8ff3-4a95-8f00-ecd21284d95e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:20.689187 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a46cc-f565-4f47-ad27-856a3f2fd0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:20.691924 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a46cc-f565-4f47-ad27-856a3f2fd0d5 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:20.692810 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a46cc-f565-4f47-ad27-856a3f2fd0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:20.693492 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a46cc-f565-4f47-ad27-856a3f2fd0d5 None admin] PATH_INFO: `/v3/projects/9799d178c2a042baa9963766f953c6a6` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:20.695934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a46cc-f565-4f47-ad27-856a3f2fd0d5 None admin] RBAC: Authorizing `identity:get_project(project_id=9799d178c2a042baa9963766f953c6a6)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:20.699931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a46cc-f565-4f47-ad27-856a3f2fd0d5 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:20.704783 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 210/431] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:41:20 2022] GET /identity/v3/projects/9799d178c2a042baa9963766f953c6a6 => generated 298 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:20.710847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a46cc-f565-4f47-ad27-856a3f2fd0d5 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:20.757029 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75589c13-e1d0-4ed1-956b-09b3a56f6867 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:20.842351 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c01843-ca7c-4768-920f-ee9995463320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:20.844730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c01843-ca7c-4768-920f-ee9995463320 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:20.845444 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c01843-ca7c-4768-920f-ee9995463320 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:20.846981 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c01843-ca7c-4768-920f-ee9995463320 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:20.874776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c01843-ca7c-4768-920f-ee9995463320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:20.896575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-99c01843-ca7c-4768-920f-ee9995463320 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:41:20.896575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:20.896575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:20.896575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:20.896575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:20.896575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:20.896575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:20.896575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:20.896575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:20.896575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:20.896575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:20.896575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:20.896575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:20.896575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:20.896575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:20.896575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:20.898697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:20.898697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:20.898697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:20.898697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:20.898697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:20.898697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:20.898697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:20.898697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:20.898697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:20.898697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:20.898697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:20.898697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:20.898697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:20.898697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:20.898697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:20.898697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:20.900098 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:20.900098 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:20.900098 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:20.900098 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:20.900098 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:20.900098 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:20.900098 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:20.900098 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:20.900098 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:20.900098 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:20.900098 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:41:20.900098 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:20.904209 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 222/432] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:41:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:41:20.914100 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c01843-ca7c-4768-920f-ee9995463320 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:20.955291 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a46cc-f565-4f47-ad27-856a3f2fd0d5 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:21.012601 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:21.021626 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a3d0f-de21-4b17-9095-a189e3455dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:21.023367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a3d0f-de21-4b17-9095-a189e3455dce None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:21.023969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a3d0f-de21-4b17-9095-a189e3455dce None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:21.024526 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a3d0f-de21-4b17-9095-a189e3455dce None admin] PATH_INFO: `/v3/groups/59d498c7d42543ad9a0d6568a293c893` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:21.026939 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a3d0f-de21-4b17-9095-a189e3455dce None admin] RBAC: Authorizing `identity:get_group(group_id=59d498c7d42543ad9a0d6568a293c893)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:21.029753 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a3d0f-de21-4b17-9095-a189e3455dce None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:21.034523 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 211/433] 10.222.0.78 () {56 vars in 1180 bytes} [Wed May 25 14:41:20 2022] GET /identity/v3/groups/59d498c7d42543ad9a0d6568a293c893 => generated 226 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:21.040355 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a3d0f-de21-4b17-9095-a189e3455dce None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:21.078060 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c01843-ca7c-4768-920f-ee9995463320 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:21.093462 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:21.134808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4100c-f8a2-4ab3-a055-cbece8c67934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:21.135493 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4100c-f8a2-4ab3-a055-cbece8c67934 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:21.135771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4100c-f8a2-4ab3-a055-cbece8c67934 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:21.136076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4100c-f8a2-4ab3-a055-cbece8c67934 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:21.137609 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4100c-f8a2-4ab3-a055-cbece8c67934 None admin] RBAC: Adding query filter params (name=member) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:21.137914 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4100c-f8a2-4ab3-a055-cbece8c67934 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:21.139391 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4100c-f8a2-4ab3-a055-cbece8c67934 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:21.152658 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 223/434] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:41:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:21.163670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4100c-f8a2-4ab3-a055-cbece8c67934 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:21.279539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a3d0f-de21-4b17-9095-a189e3455dce None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:21.311274 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4100c-f8a2-4ab3-a055-cbece8c67934 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:21.338140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:21.351509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a1179-c41d-4aaf-9822-5181e475002a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:21.352158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a1179-c41d-4aaf-9822-5181e475002a None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:21.352434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a1179-c41d-4aaf-9822-5181e475002a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:21.352714 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a1179-c41d-4aaf-9822-5181e475002a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:21.356009 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5befa-3e89-4596-aee6-7804c9a58634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:21.357309 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5befa-3e89-4596-aee6-7804c9a58634 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:21.357832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5befa-3e89-4596-aee6-7804c9a58634 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:21.358384 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5befa-3e89-4596-aee6-7804c9a58634 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:21.360708 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5befa-3e89-4596-aee6-7804c9a58634 None admin] RBAC: Adding query filter params (group.id=59d498c7d42543ad9a0d6568a293c893, role.id=d451467ccbbf4149abbfc9b0f76b9339, scope.project.id=9799d178c2a042baa9963766f953c6a6) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:21.361304 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5befa-3e89-4596-aee6-7804c9a58634 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:21.364829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5befa-3e89-4596-aee6-7804c9a58634 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:21.379984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 212/435] 10.222.0.78 () {56 vars in 1376 bytes} [Wed May 25 14:41:21 2022] GET /identity/v3/role_assignments?group.id=59d498c7d42543ad9a0d6568a293c893&scope.project.id=9799d178c2a042baa9963766f953c6a6&role.id=d451467ccbbf4149abbfc9b0f76b9339 => generated 611 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:21.383177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238d4b8-2fca-4306-bc9f-3b8aad201117 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:21.384084 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238d4b8-2fca-4306-bc9f-3b8aad201117 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:21.384641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238d4b8-2fca-4306-bc9f-3b8aad201117 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:21.387818 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 213/436] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:21.398594 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d67b3b-a992-4045-ac10-df8dbbf09311 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:21.399055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d67b3b-a992-4045-ac10-df8dbbf09311 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:21.399558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d67b3b-a992-4045-ac10-df8dbbf09311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:21.432314 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-02d67b3b-a992-4045-ac10-df8dbbf09311 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:21.434771 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-02d67b3b-a992-4045-ac10-df8dbbf09311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:21.455803 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a1179-c41d-4aaf-9822-5181e475002a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:21.457785 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a1179-c41d-4aaf-9822-5181e475002a None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:21.592746 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 214/437] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:21.597389 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b991e0e-074f-473a-8ae0-671dec383993 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:21.597671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b991e0e-074f-473a-8ae0-671dec383993 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:21.597956 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b991e0e-074f-473a-8ae0-671dec383993 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:21.599777 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 215/438] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:21.605638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e7c32-4d2b-41d6-8f1b-b2495a42af5d None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:21.605944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e7c32-4d2b-41d6-8f1b-b2495a42af5d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:21.606254 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e7c32-4d2b-41d6-8f1b-b2495a42af5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:21.630126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-e95e7c32-4d2b-41d6-8f1b-b2495a42af5d None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:21.631439 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e95e7c32-4d2b-41d6-8f1b-b2495a42af5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:21.644905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:21.645099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 224/439] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:41:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 14:41:21.645898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a1179-c41d-4aaf-9822-5181e475002a None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:21.677918 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:21.719009 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 216/440] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:41:21.724306 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e7c32-4d2b-41d6-8f1b-b2495a42af5d None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:21.771420 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a1179-c41d-4aaf-9822-5181e475002a None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:21.819023 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32231b-d8bd-4227-a66c-a3a8ac831249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:21.819855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32231b-d8bd-4227-a66c-a3a8ac831249 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:21.820130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32231b-d8bd-4227-a66c-a3a8ac831249 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:21.820397 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32231b-d8bd-4227-a66c-a3a8ac831249 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:21.833108 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d32231b-d8bd-4227-a66c-a3a8ac831249 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:21.841500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-0d32231b-d8bd-4227-a66c-a3a8ac831249 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:41:21.841500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:21.841500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:21.841500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:21.841500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:21.841500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:21.841500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:21.841500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:21.841500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:21.841500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:21.841500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:21.841500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:21.841500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:21.841500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:21.841500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:21.841500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:21.842607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:21.842607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:21.842607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:21.842607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:21.842607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:21.842607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:21.842607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:21.842607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:21.842607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:21.842607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:21.842607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:21.842607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:21.842607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:21.842607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:21.842607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:21.842607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:21.843453 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:21.843453 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:21.843453 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:21.843453 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:21.843453 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:21.843453 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:21.843453 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:21.843453 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:21.843453 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:21.843453 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:21.843453 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:41:21.843453 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:21.843453 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 225/441] 10.222.0.78 () {56 vars in 1114 bytes} [Wed May 25 14:41:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:21.845356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32231b-d8bd-4227-a66c-a3a8ac831249 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:21.868046 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e7c32-4d2b-41d6-8f1b-b2495a42af5d None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:21.918346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752aff14-01ce-49d7-9b52-35fda3d785ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:21.919690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752aff14-01ce-49d7-9b52-35fda3d785ac None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:21.920130 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752aff14-01ce-49d7-9b52-35fda3d785ac None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:21.920569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752aff14-01ce-49d7-9b52-35fda3d785ac None admin] PATH_INFO: `/v3/projects/212e33252b164b33ae38cf0925858a09` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:21.922621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752aff14-01ce-49d7-9b52-35fda3d785ac None admin] RBAC: Authorizing `identity:get_project(project_id=212e33252b164b33ae38cf0925858a09)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:21.925082 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752aff14-01ce-49d7-9b52-35fda3d785ac None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:21.928236 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 217/442] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:41:21 2022] GET /identity/v3/projects/212e33252b164b33ae38cf0925858a09 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:21.929568 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752aff14-01ce-49d7-9b52-35fda3d785ac None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:21.956268 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:21.958118 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:21.978337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32231b-d8bd-4227-a66c-a3a8ac831249 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:22.019997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd891a-5894-4fe9-87d3-b5c1418995ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:22.020794 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd891a-5894-4fe9-87d3-b5c1418995ae None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:22.021070 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd891a-5894-4fe9-87d3-b5c1418995ae None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:22.021340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd891a-5894-4fe9-87d3-b5c1418995ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:22.032875 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd891a-5894-4fe9-87d3-b5c1418995ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:22.040846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-6bbd891a-5894-4fe9-87d3-b5c1418995ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:41:22.040846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:22.040846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:22.040846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:22.040846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:22.040846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:22.040846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:22.040846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:22.040846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:22.040846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:22.040846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:22.040846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:22.040846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:22.040846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:22.040846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:22.040846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:22.042382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:22.042382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:22.042382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:22.042382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:22.042382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:22.042382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:22.042382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:22.042382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:22.042382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:22.042382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:22.042382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:22.042382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:22.042382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:22.042382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:22.042382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:22.042382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:22.043533 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:22.043533 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:22.043533 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:22.043533 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:22.043533 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:22.043533 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:22.043533 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:22.043533 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:22.043533 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:22.043533 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:22.043533 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:41:22.043533 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:22.043533 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 226/443] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:41:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:22.043533 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd891a-5894-4fe9-87d3-b5c1418995ae None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:22.064590 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752aff14-01ce-49d7-9b52-35fda3d785ac None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:22.100062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff8dd3-e363-4bcd-9022-cfff5a068edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:22.100656 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff8dd3-e363-4bcd-9022-cfff5a068edc None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:22.100939 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff8dd3-e363-4bcd-9022-cfff5a068edc None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:22.101214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff8dd3-e363-4bcd-9022-cfff5a068edc None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:22.101961 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff8dd3-e363-4bcd-9022-cfff5a068edc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:22.102298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff8dd3-e363-4bcd-9022-cfff5a068edc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:22.103708 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff8dd3-e363-4bcd-9022-cfff5a068edc None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:22.112445 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 218/444] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:41:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:22.113402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff8dd3-e363-4bcd-9022-cfff5a068edc None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:22.151534 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd891a-5894-4fe9-87d3-b5c1418995ae None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:22.186949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a19965-6775-461e-a4f8-f6a1b08ed36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:22.188345 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a19965-6775-461e-a4f8-f6a1b08ed36e None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:22.188618 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a19965-6775-461e-a4f8-f6a1b08ed36e None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:22.188890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a19965-6775-461e-a4f8-f6a1b08ed36e None admin] PATH_INFO: `/v3/groups/59d498c7d42543ad9a0d6568a293c893` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:22.190122 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a19965-6775-461e-a4f8-f6a1b08ed36e None admin] RBAC: Authorizing `identity:get_group(group_id=59d498c7d42543ad9a0d6568a293c893)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:22.191598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a19965-6775-461e-a4f8-f6a1b08ed36e None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:22.193230 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 227/445] 10.222.0.78 () {56 vars in 1180 bytes} [Wed May 25 14:41:22 2022] GET /identity/v3/groups/59d498c7d42543ad9a0d6568a293c893 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:22.194005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a19965-6775-461e-a4f8-f6a1b08ed36e None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:22.195798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:22.211381 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:22.227872 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff8dd3-e363-4bcd-9022-cfff5a068edc None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:22.264688 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80934424-f701-4aed-b618-c27cf272db58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:22.265374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80934424-f701-4aed-b618-c27cf272db58 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:22.266226 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80934424-f701-4aed-b618-c27cf272db58 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:22.266505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80934424-f701-4aed-b618-c27cf272db58 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:22.267284 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80934424-f701-4aed-b618-c27cf272db58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:22.267598 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80934424-f701-4aed-b618-c27cf272db58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:22.268993 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80934424-f701-4aed-b618-c27cf272db58 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:22.278839 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 219/446] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:41:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:22.279678 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80934424-f701-4aed-b618-c27cf272db58 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:22.303670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a19965-6775-461e-a4f8-f6a1b08ed36e None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:22.342180 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f0714-cbf1-4e05-b7cf-331677050a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:22.342828 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f0714-cbf1-4e05-b7cf-331677050a2d None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:22.343111 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f0714-cbf1-4e05-b7cf-331677050a2d None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:22.343388 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f0714-cbf1-4e05-b7cf-331677050a2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:22.384671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80934424-f701-4aed-b618-c27cf272db58 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:22.413232 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f0714-cbf1-4e05-b7cf-331677050a2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:22.414739 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f0714-cbf1-4e05-b7cf-331677050a2d None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:22.419324 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6967864-8c04-4ec2-a47a-37ecf49f3732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:22.419977 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6967864-8c04-4ec2-a47a-37ecf49f3732 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:22.420266 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6967864-8c04-4ec2-a47a-37ecf49f3732 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:22.420548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6967864-8c04-4ec2-a47a-37ecf49f3732 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:22.421372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6967864-8c04-4ec2-a47a-37ecf49f3732 None admin] RBAC: Adding query filter params (group.id=59d498c7d42543ad9a0d6568a293c893, role.id=a3da3c62738d4790b9dac512b1568b52, scope.project.id=212e33252b164b33ae38cf0925858a09) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:22.421688 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6967864-8c04-4ec2-a47a-37ecf49f3732 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:22.423230 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6967864-8c04-4ec2-a47a-37ecf49f3732 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:22.426499 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:22.426776 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:22.430714 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 220/447] 10.222.0.78 () {56 vars in 1376 bytes} [Wed May 25 14:41:22 2022] GET /identity/v3/role_assignments?group.id=59d498c7d42543ad9a0d6568a293c893&scope.project.id=212e33252b164b33ae38cf0925858a09&role.id=a3da3c62738d4790b9dac512b1568b52 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:22.431844 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6967864-8c04-4ec2-a47a-37ecf49f3732 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:22.557483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 228/448] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:41:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:41:22.559955 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6967864-8c04-4ec2-a47a-37ecf49f3732 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:22.564791 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f0714-cbf1-4e05-b7cf-331677050a2d None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:22.610614 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e6570-3cef-4c88-aca1-61ab8495cdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:22.611338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e6570-3cef-4c88-aca1-61ab8495cdff None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:22.611575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e6570-3cef-4c88-aca1-61ab8495cdff None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:22.611860 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e6570-3cef-4c88-aca1-61ab8495cdff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:22.694026 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:22.698387 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e6570-3cef-4c88-aca1-61ab8495cdff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:22.698387 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e6570-3cef-4c88-aca1-61ab8495cdff None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:22.709335 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f0714-cbf1-4e05-b7cf-331677050a2d None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:22.723925 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:22.753778 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86a619-b532-48d8-bbe8-ce8342509111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:22.754686 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86a619-b532-48d8-bbe8-ce8342509111 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:22.754973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86a619-b532-48d8-bbe8-ce8342509111 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:22.755252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86a619-b532-48d8-bbe8-ce8342509111 None admin] PATH_INFO: `/v3/projects/212e33252b164b33ae38cf0925858a09` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:22.757578 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86a619-b532-48d8-bbe8-ce8342509111 None admin] RBAC: Authorizing `identity:get_project(project_id=212e33252b164b33ae38cf0925858a09)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:22.758835 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86a619-b532-48d8-bbe8-ce8342509111 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:22.761755 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 229/449] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:41:22 2022] GET /identity/v3/projects/212e33252b164b33ae38cf0925858a09 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:22.766526 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86a619-b532-48d8-bbe8-ce8342509111 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:22.825127 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 221/450] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:41:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:41:22.826419 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e6570-3cef-4c88-aca1-61ab8495cdff None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:22.890328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86a619-b532-48d8-bbe8-ce8342509111 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:22.927220 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618bccf3-277f-478f-9923-6d80a63c422e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:22.928079 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bccf3-277f-478f-9923-6d80a63c422e None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:22.928362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bccf3-277f-478f-9923-6d80a63c422e None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:22.928641 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bccf3-277f-478f-9923-6d80a63c422e None admin] PATH_INFO: `/v3/groups/59d498c7d42543ad9a0d6568a293c893` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:22.929836 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618bccf3-277f-478f-9923-6d80a63c422e None admin] RBAC: Authorizing `identity:get_group(group_id=59d498c7d42543ad9a0d6568a293c893)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:22.931007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:22.931312 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618bccf3-277f-478f-9923-6d80a63c422e None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:22.933159 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 230/451] 10.222.0.78 () {56 vars in 1180 bytes} [Wed May 25 14:41:22 2022] GET /identity/v3/groups/59d498c7d42543ad9a0d6568a293c893 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:22.934066 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e6570-3cef-4c88-aca1-61ab8495cdff None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:22.940420 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618bccf3-277f-478f-9923-6d80a63c422e None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:22.970820 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad7507-904d-453a-88bc-40f25cff9288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:22.971581 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad7507-904d-453a-88bc-40f25cff9288 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:22.971860 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad7507-904d-453a-88bc-40f25cff9288 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:22.972131 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad7507-904d-453a-88bc-40f25cff9288 None admin] PATH_INFO: `/v3/projects/86b029bd8d8948dfab182a990de56b86` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:22.973305 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad7507-904d-453a-88bc-40f25cff9288 None admin] RBAC: Authorizing `identity:get_project(project_id=86b029bd8d8948dfab182a990de56b86)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:22.974785 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad7507-904d-453a-88bc-40f25cff9288 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:22.976468 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 222/452] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:41:22 2022] GET /identity/v3/projects/86b029bd8d8948dfab182a990de56b86 => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:22.981815 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad7507-904d-453a-88bc-40f25cff9288 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:22.999725 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:23.066323 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618bccf3-277f-478f-9923-6d80a63c422e None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:23.090298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad7507-904d-453a-88bc-40f25cff9288 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:23.103466 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfce9a6-8a29-4104-877b-fa1856a3551a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:23.104480 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfce9a6-8a29-4104-877b-fa1856a3551a None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:23.104761 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfce9a6-8a29-4104-877b-fa1856a3551a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:23.105041 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfce9a6-8a29-4104-877b-fa1856a3551a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:23.105992 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfce9a6-8a29-4104-877b-fa1856a3551a None admin] RBAC: Adding query filter params (group.id=59d498c7d42543ad9a0d6568a293c893, role.id=d451467ccbbf4149abbfc9b0f76b9339, scope.project.id=212e33252b164b33ae38cf0925858a09) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:23.106223 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfce9a6-8a29-4104-877b-fa1856a3551a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:23.107690 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfce9a6-8a29-4104-877b-fa1856a3551a None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:23.114790 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 231/453] 10.222.0.78 () {56 vars in 1376 bytes} [Wed May 25 14:41:22 2022] GET /identity/v3/role_assignments?group.id=59d498c7d42543ad9a0d6568a293c893&scope.project.id=212e33252b164b33ae38cf0925858a09&role.id=d451467ccbbf4149abbfc9b0f76b9339 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:23.133934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318184b9-b99c-450d-92f7-c71b5e5d6f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:23.134872 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318184b9-b99c-450d-92f7-c71b5e5d6f6a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:23.135156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318184b9-b99c-450d-92f7-c71b5e5d6f6a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:23.135432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318184b9-b99c-450d-92f7-c71b5e5d6f6a None admin] PATH_INFO: `/v3/groups/463a21f48e854817af55cb2d80c448bf` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:23.136775 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318184b9-b99c-450d-92f7-c71b5e5d6f6a None admin] RBAC: Authorizing `identity:get_group(group_id=463a21f48e854817af55cb2d80c448bf)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:23.138314 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318184b9-b99c-450d-92f7-c71b5e5d6f6a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:23.140090 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 223/454] 10.222.0.78 () {56 vars in 1180 bytes} [Wed May 25 14:41:22 2022] GET /identity/v3/groups/463a21f48e854817af55cb2d80c448bf => generated 229 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:23.146804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfce9a6-8a29-4104-877b-fa1856a3551a None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:23.260006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfce9a6-8a29-4104-877b-fa1856a3551a None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:23.272323 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:23.299499 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8546d3-d8b9-4aa2-aebb-fe4d01c27187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:23.300162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8546d3-d8b9-4aa2-aebb-fe4d01c27187 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:23.300444 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8546d3-d8b9-4aa2-aebb-fe4d01c27187 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:23.300728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8546d3-d8b9-4aa2-aebb-fe4d01c27187 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:23.301566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8546d3-d8b9-4aa2-aebb-fe4d01c27187 None admin] RBAC: Adding query filter params (group.id=463a21f48e854817af55cb2d80c448bf, role.id=e566b37cd92f467198b5dc40f183effc, scope.project.id=86b029bd8d8948dfab182a990de56b86) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:23.301879 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8546d3-d8b9-4aa2-aebb-fe4d01c27187 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:23.303353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8546d3-d8b9-4aa2-aebb-fe4d01c27187 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:23.309738 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 232/455] 10.222.0.78 () {56 vars in 1376 bytes} [Wed May 25 14:41:23 2022] GET /identity/v3/role_assignments?group.id=463a21f48e854817af55cb2d80c448bf&scope.project.id=86b029bd8d8948dfab182a990de56b86&role.id=e566b37cd92f467198b5dc40f183effc => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:25.950418 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d55777-6b12-48c4-87d3-2d8a5cbd52e3 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:25.950933 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d55777-6b12-48c4-87d3-2d8a5cbd52e3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:25.951058 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d55777-6b12-48c4-87d3-2d8a5cbd52e3 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:25.954372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 224/456] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:25.958324 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb48e4-f734-4589-8d15-f7cd7af9c3bc None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:25.958816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb48e4-f734-4589-8d15-f7cd7af9c3bc None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:25.958892 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb48e4-f734-4589-8d15-f7cd7af9c3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:25.983719 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-99fb48e4-f734-4589-8d15-f7cd7af9c3bc None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:25.986391 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-99fb48e4-f734-4589-8d15-f7cd7af9c3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:26.116361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 233/457] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:26.160920 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d55777-6b12-48c4-87d3-2d8a5cbd52e3 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:26.227370 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:26.355880 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d55777-6b12-48c4-87d3-2d8a5cbd52e3 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:26.401408 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4debf-0ac0-4ea6-9b07-43d70a259655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:26.402702 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4debf-0ac0-4ea6-9b07-43d70a259655 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:26.403207 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4debf-0ac0-4ea6-9b07-43d70a259655 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:26.403692 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4debf-0ac0-4ea6-9b07-43d70a259655 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:26.419658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4debf-0ac0-4ea6-9b07-43d70a259655 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:26.421568 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4debf-0ac0-4ea6-9b07-43d70a259655 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:26.451340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-94e4debf-0ac0-4ea6-9b07-43d70a259655 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:26.451340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:26.451340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:26.451340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:26.451340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:26.451340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:26.451340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:26.451340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:26.451340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:26.451340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:26.451340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:26.451340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:26.451340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:26.451340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:26.451340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:26.452265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:26.452265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:26.452265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:26.452265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:26.452265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:26.452265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:26.452265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:26.452265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:26.452265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:26.452265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:26.452265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:26.452265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:26.452265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:26.452265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:26.452265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:26.452265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:26.452265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:26.453185 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:26.453185 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:26.453185 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:26.453185 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:26.459765 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 225/458] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:26.467184 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb48e4-f734-4589-8d15-f7cd7af9c3bc None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:26.503934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cd8af-8cd0-422f-b346-6a1ffe4e11e2 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:26.504562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cd8af-8cd0-422f-b346-6a1ffe4e11e2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:26.505063 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cd8af-8cd0-422f-b346-6a1ffe4e11e2 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:26.506965 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 226/459] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:41:26.513080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169f915-5491-41c2-96ed-956b0ce642c7 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:26.513689 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169f915-5491-41c2-96ed-956b0ce642c7 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:26.514238 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169f915-5491-41c2-96ed-956b0ce642c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:26.556184 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-a169f915-5491-41c2-96ed-956b0ce642c7 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:26.557834 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a169f915-5491-41c2-96ed-956b0ce642c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:26.709087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb48e4-f734-4589-8d15-f7cd7af9c3bc None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:26.713940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:26.722711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 227/460] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:41:26.728831 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13310168-49d0-49a5-8c5e-20eec9fa26f1 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:26.728831 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13310168-49d0-49a5-8c5e-20eec9fa26f1 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:26.728831 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13310168-49d0-49a5-8c5e-20eec9fa26f1 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:26.739641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 228/461] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:26 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:26.744230 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b881729-98b1-4c29-8360-7c27cab6aa02 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:26.744735 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b881729-98b1-4c29-8360-7c27cab6aa02 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:26.745234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b881729-98b1-4c29-8360-7c27cab6aa02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:26.773856 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-8b881729-98b1-4c29-8360-7c27cab6aa02 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:26.775110 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8b881729-98b1-4c29-8360-7c27cab6aa02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:26.789903 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e88c8d-56f3-497d-a08c-00b60b58be5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:26.791083 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e88c8d-56f3-497d-a08c-00b60b58be5a None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:26.791575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e88c8d-56f3-497d-a08c-00b60b58be5a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:26.792058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e88c8d-56f3-497d-a08c-00b60b58be5a None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:26.792742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:26.793420 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e88c8d-56f3-497d-a08c-00b60b58be5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:26.794003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e88c8d-56f3-497d-a08c-00b60b58be5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:26.797199 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e88c8d-56f3-497d-a08c-00b60b58be5a None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:26.824939 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 234/462] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:26.831708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e88c8d-56f3-497d-a08c-00b60b58be5a None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:26.865125 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 229/463] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:41:26.866495 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818f802-c80d-4466-a0ab-c7e4aa05a0e8 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:26.866789 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818f802-c80d-4466-a0ab-c7e4aa05a0e8 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:26.867066 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818f802-c80d-4466-a0ab-c7e4aa05a0e8 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:26.868570 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 230/464] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:26.869050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818f802-c80d-4466-a0ab-c7e4aa05a0e8 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:26.966580 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e88c8d-56f3-497d-a08c-00b60b58be5a None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:27.005060 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818f802-c80d-4466-a0ab-c7e4aa05a0e8 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:27.016111 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08227c7-5ca5-42c1-903a-17b14be1501d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:27.016836 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08227c7-5ca5-42c1-903a-17b14be1501d None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:27.016947 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08227c7-5ca5-42c1-903a-17b14be1501d None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:27.020912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08227c7-5ca5-42c1-903a-17b14be1501d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:27.035003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08227c7-5ca5-42c1-903a-17b14be1501d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:27.036656 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08227c7-5ca5-42c1-903a-17b14be1501d None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:27.059729 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f2628-a828-46d5-bb0b-89d8cf7cb14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:27.060319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-e08227c7-5ca5-42c1-903a-17b14be1501d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:27.060319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:27.060319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:27.060319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:27.060319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:27.060319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:27.060319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:27.060319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:27.060319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:27.060319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:27.060319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:27.060319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:27.060319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:27.060319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:27.060319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:27.061286 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:27.061286 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:27.061286 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:27.061286 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:27.061286 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:27.061286 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:27.061286 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:27.061286 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:27.061286 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:27.061286 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:27.061286 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:27.061286 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:27.061286 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:27.061286 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:27.061286 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:27.061286 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:27.061286 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:27.069125 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:27.069125 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:27.069125 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:27.069125 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:27.069514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f2628-a828-46d5-bb0b-89d8cf7cb14f None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:27.069514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f2628-a828-46d5-bb0b-89d8cf7cb14f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:27.069514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f2628-a828-46d5-bb0b-89d8cf7cb14f None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:27.069844 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 235/465] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:27.069952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f2628-a828-46d5-bb0b-89d8cf7cb14f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:27.070095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc937014-2bfc-4bff-bd58-fee95e3bd0cf None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:27.070095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc937014-2bfc-4bff-bd58-fee95e3bd0cf None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:27.070095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc937014-2bfc-4bff-bd58-fee95e3bd0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:27.070408 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f2628-a828-46d5-bb0b-89d8cf7cb14f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:27.090456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-fc937014-2bfc-4bff-bd58-fee95e3bd0cf None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:27.091655 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-fc937014-2bfc-4bff-bd58-fee95e3bd0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:27.102112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 231/466] 10.222.0.78 () {60 vars in 1132 bytes} [Wed May 25 14:41:26 2022] POST /identity/v3/users => generated 246 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:27.102112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f2628-a828-46d5-bb0b-89d8cf7cb14f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:27.106538 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:27.116308 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:27.206448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 236/467] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:27.207662 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc937014-2bfc-4bff-bd58-fee95e3bd0cf None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:27.284504 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f2628-a828-46d5-bb0b-89d8cf7cb14f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:27.358679 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8348030-595e-451c-97e0-7fd3d651344d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:27.360169 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8348030-595e-451c-97e0-7fd3d651344d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:27.360449 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8348030-595e-451c-97e0-7fd3d651344d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:27.360724 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8348030-595e-451c-97e0-7fd3d651344d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:27.376909 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8348030-595e-451c-97e0-7fd3d651344d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:27.380865 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8348030-595e-451c-97e0-7fd3d651344d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:27.403646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-c8348030-595e-451c-97e0-7fd3d651344d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:27.403646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:27.403646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:27.403646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:27.403646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:27.403646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:27.403646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:27.403646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:27.403646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:27.403646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:27.403646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:27.403646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:27.403646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:27.403646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:27.403646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:27.404544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:27.404544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:27.404544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:27.404544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:27.404544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:27.404544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:27.404544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:27.404544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:27.404544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:27.404544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:27.404544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:27.404544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:27.404544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:27.404544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:27.404544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:27.404544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:27.404544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:27.405374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:27.405374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:27.405374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:27.405374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:27.405578 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc937014-2bfc-4bff-bd58-fee95e3bd0cf None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:27.408156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 232/468] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:27.409270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8348030-595e-451c-97e0-7fd3d651344d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:27.457797 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc4799-fc9f-46a9-928f-028afc5d4a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:27.458809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc4799-fc9f-46a9-928f-028afc5d4a09 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:27.459116 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc4799-fc9f-46a9-928f-028afc5d4a09 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:27.459566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc4799-fc9f-46a9-928f-028afc5d4a09 None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:27.460868 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc4799-fc9f-46a9-928f-028afc5d4a09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:27.461191 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc4799-fc9f-46a9-928f-028afc5d4a09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:27.462820 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc4799-fc9f-46a9-928f-028afc5d4a09 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:27.476079 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 237/469] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:27.479411 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc4799-fc9f-46a9-928f-028afc5d4a09 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:27.490446 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:27.547913 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:27.584129 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8348030-595e-451c-97e0-7fd3d651344d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:27.626324 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc4799-fc9f-46a9-928f-028afc5d4a09 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:27.648704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc17b47-e889-4c5c-8fd7-1d1f55b3c5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:27.649730 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc17b47-e889-4c5c-8fd7-1d1f55b3c5b6 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:27.650042 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc17b47-e889-4c5c-8fd7-1d1f55b3c5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:27.650312 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc17b47-e889-4c5c-8fd7-1d1f55b3c5b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:27.666515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc17b47-e889-4c5c-8fd7-1d1f55b3c5b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:27.668720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc17b47-e889-4c5c-8fd7-1d1f55b3c5b6 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:27.683719 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2952422-8683-4e06-bce7-32201930bf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:27.684447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2952422-8683-4e06-bce7-32201930bf45 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:27.684732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2952422-8683-4e06-bce7-32201930bf45 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:27.685005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2952422-8683-4e06-bce7-32201930bf45 None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:27.685765 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2952422-8683-4e06-bce7-32201930bf45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:27.686105 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2952422-8683-4e06-bce7-32201930bf45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:27.688269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2952422-8683-4e06-bce7-32201930bf45 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:27.696772 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-cfc17b47-e889-4c5c-8fd7-1d1f55b3c5b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:27.696772 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:27.696772 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:27.696772 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:27.696772 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:27.696772 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:27.696772 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:27.696772 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:27.696772 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:27.696772 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:27.696772 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:27.696772 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:27.696772 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:27.696772 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:27.696772 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:27.697677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:27.697677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:27.697677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:27.697677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:27.697677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:27.697677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:27.697677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:27.697677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:27.697677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:27.697677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:27.697677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:27.697677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:27.697677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:27.697677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:27.697677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:27.697677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:27.697677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:27.698916 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:27.698916 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:27.698916 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:27.698916 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:27.701578 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 233/470] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:27.702768 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc17b47-e889-4c5c-8fd7-1d1f55b3c5b6 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:27.708276 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 238/471] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:27.709251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2952422-8683-4e06-bce7-32201930bf45 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:27.785224 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:27.826049 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2952422-8683-4e06-bce7-32201930bf45 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:27.846165 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc17b47-e889-4c5c-8fd7-1d1f55b3c5b6 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:27.857701 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:27.867982 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42413f42-1076-4e21-ac51-45ccd80f43df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:27.868678 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42413f42-1076-4e21-ac51-45ccd80f43df None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:27.869001 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42413f42-1076-4e21-ac51-45ccd80f43df None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:27.869270 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42413f42-1076-4e21-ac51-45ccd80f43df None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:27.869945 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42413f42-1076-4e21-ac51-45ccd80f43df None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:27.870294 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42413f42-1076-4e21-ac51-45ccd80f43df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:27.871699 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42413f42-1076-4e21-ac51-45ccd80f43df None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:27.884699 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 239/472] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:27.885561 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42413f42-1076-4e21-ac51-45ccd80f43df None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:27.897177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481a24e-f107-4a92-a35f-4fc841dea697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:27.897177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481a24e-f107-4a92-a35f-4fc841dea697 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:27.897177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481a24e-f107-4a92-a35f-4fc841dea697 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:27.897177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481a24e-f107-4a92-a35f-4fc841dea697 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:27.897597 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481a24e-f107-4a92-a35f-4fc841dea697 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:27.898005 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481a24e-f107-4a92-a35f-4fc841dea697 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:27.920711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 234/473] 10.222.0.78 () {60 vars in 1132 bytes} [Wed May 25 14:41:27 2022] POST /identity/v3/users => generated 248 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:27.923034 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad4af7-5585-4cc0-9d2c-c36bfa33aae8 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:27.923327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad4af7-5585-4cc0-9d2c-c36bfa33aae8 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:27.923606 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad4af7-5585-4cc0-9d2c-c36bfa33aae8 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:27.925293 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 235/474] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:27.925839 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad4af7-5585-4cc0-9d2c-c36bfa33aae8 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:28.045145 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42413f42-1076-4e21-ac51-45ccd80f43df None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:28.083995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:28.095722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532d73a-2809-4de1-bdc8-2a34d2039f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:28.096775 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532d73a-2809-4de1-bdc8-2a34d2039f24 None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:28.097246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532d73a-2809-4de1-bdc8-2a34d2039f24 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:28.097621 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532d73a-2809-4de1-bdc8-2a34d2039f24 None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:28.097726 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad4af7-5585-4cc0-9d2c-c36bfa33aae8 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:28.098707 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532d73a-2809-4de1-bdc8-2a34d2039f24 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:28.100477 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532d73a-2809-4de1-bdc8-2a34d2039f24 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:28.129730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 240/475] 10.222.0.78 () {60 vars in 1132 bytes} [Wed May 25 14:41:27 2022] POST /identity/v3/users => generated 248 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:28.131339 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b73c9e-38e7-4f1e-8357-2c7e09a91590 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:28.131777 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b73c9e-38e7-4f1e-8357-2c7e09a91590 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:28.132272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b73c9e-38e7-4f1e-8357-2c7e09a91590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:28.153401 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55328b5-e3ab-46d3-ae79-e5ab8aa943cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:28.154259 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55328b5-e3ab-46d3-ae79-e5ab8aa943cf None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:28.154422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-85b73c9e-38e7-4f1e-8357-2c7e09a91590 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:28.154563 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55328b5-e3ab-46d3-ae79-e5ab8aa943cf None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:28.154829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55328b5-e3ab-46d3-ae79-e5ab8aa943cf None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:28.155649 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55328b5-e3ab-46d3-ae79-e5ab8aa943cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:28.156466 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-85b73c9e-38e7-4f1e-8357-2c7e09a91590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:28.157810 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55328b5-e3ab-46d3-ae79-e5ab8aa943cf None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:28.200090 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 236/476] 10.222.0.78 () {60 vars in 1132 bytes} [Wed May 25 14:41:27 2022] POST /identity/v3/users => generated 249 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:28.286206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 241/477] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:28.318792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55328b5-e3ab-46d3-ae79-e5ab8aa943cf None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:28.348541 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:28.463568 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55328b5-e3ab-46d3-ae79-e5ab8aa943cf None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:28.501231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28b9b2-ef4b-4ed9-80d6-7f022d891ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:28.501903 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28b9b2-ef4b-4ed9-80d6-7f022d891ea1 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:28.502316 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28b9b2-ef4b-4ed9-80d6-7f022d891ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:28.502655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28b9b2-ef4b-4ed9-80d6-7f022d891ea1 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:28.503429 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28b9b2-ef4b-4ed9-80d6-7f022d891ea1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:28.504872 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28b9b2-ef4b-4ed9-80d6-7f022d891ea1 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:28.513757 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.sql.core [None req-bd28b9b2-ef4b-4ed9-80d6-7f022d891ea1 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 25 14:41:28.513757 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 25 14:41:28.513757 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [parameters: {'id': '63b7a6725d0a4915877336b0f0ed7e19', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 25 14:41:28.513757 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86693) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 25 14:41:28.532156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-bd28b9b2-ef4b-4ed9-80d6-7f022d891ea1 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 25 14:41:28.532156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:28.532156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 14:41:28.532156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 14:41:28.532156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 14:41:28.532156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 14:41:28.532156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 14:41:28.532156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application result = self._query(query) May 25 14:41:28.532156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 14:41:28.532156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application conn.query(q) May 25 14:41:28.532156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 14:41:28.532156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 14:41:28.532156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 14:41:28.532156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application result.read() May 25 14:41:28.532156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 14:41:28.533078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 14:41:28.533078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 14:41:28.533078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 14:41:28.533078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 14:41:28.533078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 14:41:28.533078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 14:41:28.533078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 14:41:28.533078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 25 14:41:28.533078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:28.533078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 25 14:41:28.533078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:28.533078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:28.533078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 25 14:41:28.533078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 25 14:41:28.533078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 25 14:41:28.533078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return ref.to_dict() May 25 14:41:28.533078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 14:41:28.533078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application next(self.gen) May 25 14:41:28.534270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 25 14:41:28.534270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application yield resource May 25 14:41:28.534270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 14:41:28.534270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application next(self.gen) May 25 14:41:28.534270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 25 14:41:28.534270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self.session.rollback() May 25 14:41:28.534270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 25 14:41:28.534270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self.force_reraise() May 25 14:41:28.534270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 25 14:41:28.534270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise self.value May 25 14:41:28.534270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 25 14:41:28.534270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 25 14:41:28.534270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 25 14:41:28.534270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application session.commit() May 25 14:41:28.534270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 25 14:41:28.534270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 25 14:41:28.534270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 25 14:41:28.535301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._prepare_impl() May 25 14:41:28.535301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 25 14:41:28.535301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self.session.flush() May 25 14:41:28.535301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 25 14:41:28.535301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._flush(objects) May 25 14:41:28.535301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 25 14:41:28.535301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 25 14:41:28.535301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 25 14:41:28.535301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application compat.raise_( May 25 14:41:28.535301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 14:41:28.535301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception May 25 14:41:28.535301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 25 14:41:28.535301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application flush_context.execute() May 25 14:41:28.535301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 25 14:41:28.535301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rec.execute(self) May 25 14:41:28.535301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 25 14:41:28.535301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 25 14:41:28.536163 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 25 14:41:28.536163 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application _emit_insert_statements( May 25 14:41:28.536163 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 25 14:41:28.536163 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application c = connection._execute_20( May 25 14:41:28.536163 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 25 14:41:28.536163 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 25 14:41:28.536163 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 25 14:41:28.536163 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 25 14:41:28.536163 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 25 14:41:28.536163 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ret = self._execute_context( May 25 14:41:28.536163 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 25 14:41:28.536163 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 25 14:41:28.536163 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 25 14:41:28.536163 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 25 14:41:28.536163 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 14:41:28.536163 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception May 25 14:41:28.537208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 14:41:28.537208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 14:41:28.537208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 14:41:28.537208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 14:41:28.537208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 14:41:28.537208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application result = self._query(query) May 25 14:41:28.537208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 14:41:28.537208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application conn.query(q) May 25 14:41:28.537208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 14:41:28.537208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 14:41:28.537208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 14:41:28.537208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application result.read() May 25 14:41:28.537208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 14:41:28.537208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 14:41:28.537208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 14:41:28.537208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 14:41:28.537991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 14:41:28.537991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 14:41:28.537991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 14:41:28.537991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 14:41:28.537991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 25 14:41:28.537991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 25 14:41:28.537991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [parameters: {'id': '63b7a6725d0a4915877336b0f0ed7e19', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 25 14:41:28.537991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 25 14:41:28.537991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:28.537991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:41:28.537991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:28.537991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:28.537991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:28.537991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:28.537991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:28.537991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:28.542928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:28.542928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:28.542928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:28.542928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:28.542928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:28.542928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:28.542928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 25 14:41:28.542928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 25 14:41:28.542928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:28.542928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:28.542928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 25 14:41:28.542928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 25 14:41:28.542928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 25 14:41:28.542928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 25 14:41:28.542928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 25 14:41:28.542928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:28.544890 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 237/478] 10.222.0.78 () {60 vars in 1131 bytes} [Wed May 25 14:41:28 2022] POST /identity/v3/roles => generated 144 bytes in 216 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:28.545045 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b73c9e-38e7-4f1e-8357-2c7e09a91590 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:28.713271 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b73c9e-38e7-4f1e-8357-2c7e09a91590 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:28.732884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:28.770894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a4baa1-d514-4649-b538-c79111dcc0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:28.772104 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4baa1-d514-4649-b538-c79111dcc0ed None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:28.772608 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4baa1-d514-4649-b538-c79111dcc0ed None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:28.773077 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4baa1-d514-4649-b538-c79111dcc0ed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:28.786729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a4baa1-d514-4649-b538-c79111dcc0ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:28.797732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-51a4baa1-d514-4649-b538-c79111dcc0ed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:41:28.797732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:28.797732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:28.797732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:28.797732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:28.797732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:28.797732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:28.797732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:28.797732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:28.797732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:28.797732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:28.797732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:28.797732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:28.797732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:28.797732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:28.797732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:28.798724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:28.798724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:28.798724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:28.798724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:28.798724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:28.798724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:28.798724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:28.798724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:28.798724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:28.798724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:28.798724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:28.798724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:28.798724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:28.798724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:28.798724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:28.798724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:28.799928 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:28.799928 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:28.799928 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:28.799928 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:28.799928 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:28.799928 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:28.799928 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:28.799928 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:28.799928 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:28.799928 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:28.799928 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:41:28.799928 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:28.799928 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 242/479] 10.222.0.78 () {56 vars in 1114 bytes} [Wed May 25 14:41:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:28.805282 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28b9b2-ef4b-4ed9-80d6-7f022d891ea1 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:28.974627 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:28.998068 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28b9b2-ef4b-4ed9-80d6-7f022d891ea1 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:29.056369 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e2609-5017-414e-be55-be4284369eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:29.057058 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e2609-5017-414e-be55-be4284369eed None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:29.057341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e2609-5017-414e-be55-be4284369eed None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:29.057616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e2609-5017-414e-be55-be4284369eed None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:29.058408 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e2609-5017-414e-be55-be4284369eed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:29.058720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e2609-5017-414e-be55-be4284369eed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:29.060120 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e2609-5017-414e-be55-be4284369eed None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:29.074324 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 238/480] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:41:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:29.792855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83dbc0e-d691-4d2d-a474-ba1ed90805d9 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:29.793845 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83dbc0e-d691-4d2d-a474-ba1ed90805d9 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:29.794348 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83dbc0e-d691-4d2d-a474-ba1ed90805d9 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:29.796256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 243/481] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:29.802249 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c8bcd-b57c-4b26-85d1-aa1999a5b409 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:29.802801 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c8bcd-b57c-4b26-85d1-aa1999a5b409 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:29.802801 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c8bcd-b57c-4b26-85d1-aa1999a5b409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:29.824914 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-ad7c8bcd-b57c-4b26-85d1-aa1999a5b409 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:29.826151 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ad7c8bcd-b57c-4b26-85d1-aa1999a5b409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:29.940643 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 239/482] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:29.990123 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83dbc0e-d691-4d2d-a474-ba1ed90805d9 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:30.160604 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83dbc0e-d691-4d2d-a474-ba1ed90805d9 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:30.203508 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23bd74-a851-46c3-be08-c8454ba4df35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:30.204593 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23bd74-a851-46c3-be08-c8454ba4df35 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:30.205011 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23bd74-a851-46c3-be08-c8454ba4df35 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:30.205421 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23bd74-a851-46c3-be08-c8454ba4df35 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:30.217356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23bd74-a851-46c3-be08-c8454ba4df35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:30.226423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-1f23bd74-a851-46c3-be08-c8454ba4df35 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:30.226423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:30.226423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:30.226423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:30.226423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:30.226423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:30.226423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:30.226423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:30.226423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:30.226423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:30.226423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:30.226423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:30.226423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:30.226423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:30.226423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:30.226423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:30.227518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:30.227518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:30.227518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:30.227518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:30.227518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:30.227518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:30.227518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:30.227518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:30.227518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:30.227518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:30.227518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:30.227518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:30.227518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:30.227518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:30.227518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:30.227518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:30.228488 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:30.228488 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:30.228488 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:30.228488 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:30.228488 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:30.228488 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:30.228488 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:30.228488 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:30.228488 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:30.228488 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:30.228488 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:30.228488 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:30.230885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 244/483] 10.222.0.78 () {56 vars in 1102 bytes} [Wed May 25 14:41:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:30.239396 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c8bcd-b57c-4b26-85d1-aa1999a5b409 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:30.450464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:30.518554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c8bcd-b57c-4b26-85d1-aa1999a5b409 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:30.571880 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2bc76f-78e2-4292-acfa-defdd8aabb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:30.574449 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2bc76f-78e2-4292-acfa-defdd8aabb5a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:30.575331 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2bc76f-78e2-4292-acfa-defdd8aabb5a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:30.575986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2bc76f-78e2-4292-acfa-defdd8aabb5a None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:30.577436 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2bc76f-78e2-4292-acfa-defdd8aabb5a None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:30.578948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2bc76f-78e2-4292-acfa-defdd8aabb5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:30.580898 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2bc76f-78e2-4292-acfa-defdd8aabb5a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:30.599172 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 240/484] 10.222.0.78 () {56 vars in 1103 bytes} [Wed May 25 14:41:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:30.607586 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23bd74-a851-46c3-be08-c8454ba4df35 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:30.698650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:30.813193 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23bd74-a851-46c3-be08-c8454ba4df35 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:30.873546 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd00054-4f7c-4bd8-b1bf-a0de60642241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:30.874360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd00054-4f7c-4bd8-b1bf-a0de60642241 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:30.875412 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd00054-4f7c-4bd8-b1bf-a0de60642241 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:30.875880 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd00054-4f7c-4bd8-b1bf-a0de60642241 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:30.898998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd00054-4f7c-4bd8-b1bf-a0de60642241 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:30.900830 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd00054-4f7c-4bd8-b1bf-a0de60642241 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:30.943308 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-6cd00054-4f7c-4bd8-b1bf-a0de60642241 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:30.943308 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:30.943308 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:30.943308 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:30.943308 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:30.943308 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:30.943308 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:30.943308 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:30.943308 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:30.943308 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:30.943308 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:30.943308 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:30.943308 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:30.943308 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:30.943308 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:30.944245 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:30.944245 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:30.944245 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:30.944245 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:30.944245 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:30.944245 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:30.944245 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:30.944245 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:30.944245 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:30.944245 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:30.944245 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:30.944245 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:30.944245 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:30.944245 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:30.944245 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:30.944245 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:30.944245 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:30.945103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:30.945103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:30.945103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:30.945103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:30.945103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 245/485] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:30.951197 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2bc76f-78e2-4292-acfa-defdd8aabb5a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:31.157121 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2bc76f-78e2-4292-acfa-defdd8aabb5a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:31.221479 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042606e-4429-46fa-8aac-6ced8fffd033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:31.222226 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042606e-4429-46fa-8aac-6ced8fffd033 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:31.222518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042606e-4429-46fa-8aac-6ced8fffd033 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:31.222797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042606e-4429-46fa-8aac-6ced8fffd033 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:31.223570 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6042606e-4429-46fa-8aac-6ced8fffd033 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:31.223888 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6042606e-4429-46fa-8aac-6ced8fffd033 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:31.225332 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6042606e-4429-46fa-8aac-6ced8fffd033 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:31.235493 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603c7da-dfaf-42dd-8b3c-e7364d21c52e None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:31.236189 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603c7da-dfaf-42dd-8b3c-e7364d21c52e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:31.236706 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603c7da-dfaf-42dd-8b3c-e7364d21c52e None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:31.238464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:31.238909 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 246/486] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:31.243910 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4a4d5-78a6-40fe-9833-2d44db406ee0 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:31.244182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4a4d5-78a6-40fe-9833-2d44db406ee0 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:31.244693 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4a4d5-78a6-40fe-9833-2d44db406ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:31.249520 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 241/487] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:31.258494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042606e-4429-46fa-8aac-6ced8fffd033 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:31.271294 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-7da4a4d5-78a6-40fe-9833-2d44db406ee0 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:31.272861 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-7da4a4d5-78a6-40fe-9833-2d44db406ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:31.370323 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 247/488] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:41:31.372000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0a7f4-b376-4c3d-91e5-351cae68d48e None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:31.372294 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0a7f4-b376-4c3d-91e5-351cae68d48e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:31.372588 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0a7f4-b376-4c3d-91e5-351cae68d48e None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:31.373937 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 248/489] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:31.387303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa909d-c59e-4786-a9ca-8ad947e3735f None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:31.387938 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa909d-c59e-4786-a9ca-8ad947e3735f None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:31.388403 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa909d-c59e-4786-a9ca-8ad947e3735f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:31.394058 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042606e-4429-46fa-8aac-6ced8fffd033 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:31.408352 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-01fa909d-c59e-4786-a9ca-8ad947e3735f None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:31.411190 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-01fa909d-c59e-4786-a9ca-8ad947e3735f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:31.425002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:31.433153 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebad33-f1a5-4009-af44-70ab8ea3f3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:31.433997 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebad33-f1a5-4009-af44-70ab8ea3f3a1 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:31.434312 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebad33-f1a5-4009-af44-70ab8ea3f3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:31.434591 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebad33-f1a5-4009-af44-70ab8ea3f3a1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:31.443118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbebad33-f1a5-4009-af44-70ab8ea3f3a1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:31.444951 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbebad33-f1a5-4009-af44-70ab8ea3f3a1 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:31.464246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-bbebad33-f1a5-4009-af44-70ab8ea3f3a1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:41:31.464246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:31.464246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:31.464246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:31.464246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:31.464246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:31.464246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:31.464246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:31.464246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:31.464246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:31.464246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:31.464246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:31.464246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:41:31.464246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:41:31.464246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:41:31.464246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:41:31.465770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:31.465770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:31.465770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:41:31.465770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:31.465770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:41:31.465770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:31.465770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:31.465770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:31.465770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:31.465770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:31.465770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:31.465770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:31.465770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:31.465770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:31.465770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:31.465770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:31.465770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:31.465770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:31.467363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:31.467363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:31.467363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:41:31.467363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:41:31.467363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:41:31.467363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:41:31.467363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:41:31.467363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:41:31.467363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:41:31.467363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:31.470170 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 242/490] 10.222.0.78 () {56 vars in 1093 bytes} [Wed May 25 14:41:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:31.472343 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebad33-f1a5-4009-af44-70ab8ea3f3a1 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:31.514345 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 249/491] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:41:31.527616 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb620b0-2326-4be8-80cb-84eb14bbc9fa None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:31.527930 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb620b0-2326-4be8-80cb-84eb14bbc9fa None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:31.534884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb620b0-2326-4be8-80cb-84eb14bbc9fa None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:31.534884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 250/492] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:31 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:31.534884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb620b0-2326-4be8-80cb-84eb14bbc9fa None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:31.567893 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:31.620195 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebad33-f1a5-4009-af44-70ab8ea3f3a1 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:31.662903 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0e6ed-ece4-4080-b7f2-28eb180069f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:31.663629 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0e6ed-ece4-4080-b7f2-28eb180069f6 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:31.663907 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0e6ed-ece4-4080-b7f2-28eb180069f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:31.664178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0e6ed-ece4-4080-b7f2-28eb180069f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:31.676087 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d0e6ed-ece4-4080-b7f2-28eb180069f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:31.683989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-49d0e6ed-ece4-4080-b7f2-28eb180069f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:31.683989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:31.683989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:31.683989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:31.683989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:31.683989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:31.683989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:31.683989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:31.683989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:31.683989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:31.683989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:31.683989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:31.683989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:31.683989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:31.683989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:31.683989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:31.684921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:31.684921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:31.684921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:31.684921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:31.684921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:31.684921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:31.684921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:31.684921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:31.684921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:31.684921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:31.684921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:31.684921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:31.684921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:31.684921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:31.684921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:31.684921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:31.685723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:31.685723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:31.685723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:31.685723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:31.685723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:31.685723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:31.685723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:31.685723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:31.685723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:31.685723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:31.685723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:31.685723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:31.686911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb620b0-2326-4be8-80cb-84eb14bbc9fa None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:31.690606 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 243/493] 10.222.0.78 () {56 vars in 1102 bytes} [Wed May 25 14:41:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:31.692049 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f3491-891e-47c9-9149-2aebd2cb1b9b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:31.692486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f3491-891e-47c9-9149-2aebd2cb1b9b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:31.692655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f3491-891e-47c9-9149-2aebd2cb1b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:31.710627 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-c04f3491-891e-47c9-9149-2aebd2cb1b9b None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:31.711839 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c04f3491-891e-47c9-9149-2aebd2cb1b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:31.733501 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db23df-2fca-4c8b-94de-99d13e2087c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:31.734282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db23df-2fca-4c8b-94de-99d13e2087c4 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:31.734553 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db23df-2fca-4c8b-94de-99d13e2087c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:31.734983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db23df-2fca-4c8b-94de-99d13e2087c4 None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:31.735831 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db23df-2fca-4c8b-94de-99d13e2087c4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:31.736127 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db23df-2fca-4c8b-94de-99d13e2087c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:31.737610 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db23df-2fca-4c8b-94de-99d13e2087c4 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:31.779217 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 251/494] 10.222.0.78 () {56 vars in 1133 bytes} [Wed May 25 14:41:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:31.780083 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db23df-2fca-4c8b-94de-99d13e2087c4 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:31.789421 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:31.810252 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 244/495] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:31.811608 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f3491-891e-47c9-9149-2aebd2cb1b9b None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:31.887515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:31.935365 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db23df-2fca-4c8b-94de-99d13e2087c4 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:31.957245 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f3491-891e-47c9-9149-2aebd2cb1b9b None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:31.980122 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84acb9f-e52d-446e-8273-d2469fd91324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:31.980930 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84acb9f-e52d-446e-8273-d2469fd91324 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:31.981211 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84acb9f-e52d-446e-8273-d2469fd91324 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:31.981485 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84acb9f-e52d-446e-8273-d2469fd91324 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:31.995119 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84acb9f-e52d-446e-8273-d2469fd91324 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:32.003948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f566b-4d62-4a3a-bd1c-ff8ca71f8ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:32.004129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-a84acb9f-e52d-446e-8273-d2469fd91324 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:32.004129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:32.004129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:32.004129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:32.004129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:32.004129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:32.004129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:32.004129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:32.004129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:32.004129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:32.004129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:32.004129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:32.004129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:32.004129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:32.004129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:32.004129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:32.004993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:32.004993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:32.004993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:32.004993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:32.004993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:32.004993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:32.004993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:32.004993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:32.004993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:32.004993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:32.004993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:32.004993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:32.004993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:32.004993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:32.004993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:32.004993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:32.005766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:32.005766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:32.005766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:32.005766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:32.005766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:32.005766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:32.005766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:32.005766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:32.005766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:32.005766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:32.005766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:32.005766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:32.007206 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f566b-4d62-4a3a-bd1c-ff8ca71f8ba8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:32.007206 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f566b-4d62-4a3a-bd1c-ff8ca71f8ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:32.007206 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f566b-4d62-4a3a-bd1c-ff8ca71f8ba8 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:32.007407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 252/496] 10.222.0.78 () {56 vars in 1102 bytes} [Wed May 25 14:41:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:32.007472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f566b-4d62-4a3a-bd1c-ff8ca71f8ba8 None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:32.007537 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84acb9f-e52d-446e-8273-d2469fd91324 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:32.010786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f566b-4d62-4a3a-bd1c-ff8ca71f8ba8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:32.012347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f566b-4d62-4a3a-bd1c-ff8ca71f8ba8 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:32.022881 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 245/497] 10.222.0.78 () {56 vars in 1103 bytes} [Wed May 25 14:41:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:32.028294 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f566b-4d62-4a3a-bd1c-ff8ca71f8ba8 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:32.106660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:32.172530 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f566b-4d62-4a3a-bd1c-ff8ca71f8ba8 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:32.184697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:32.186479 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84acb9f-e52d-446e-8273-d2469fd91324 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:32.213810 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c54407-169b-47e5-b09f-6017ec0e647e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:32.214652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c54407-169b-47e5-b09f-6017ec0e647e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:32.215099 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c54407-169b-47e5-b09f-6017ec0e647e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:32.215190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c54407-169b-47e5-b09f-6017ec0e647e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:32.228132 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c54407-169b-47e5-b09f-6017ec0e647e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:32.236438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-15c54407-169b-47e5-b09f-6017ec0e647e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:32.236438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:32.236438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:32.236438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:32.236438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:32.236438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:32.236438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:32.236438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:32.236438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:32.236438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:32.236438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:32.236438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:32.236438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:32.236438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:32.236438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:32.236438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:32.237334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:32.237334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:32.237334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:32.237334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:32.237334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:32.237334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:32.237334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:32.237334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:32.237334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:32.237334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:32.237334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:32.237334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:32.237334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:32.237334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:32.237334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:32.237334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:32.238158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:32.238158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:32.238158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:32.238158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:32.238158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:32.238158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:32.238158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:32.238158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:32.238158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:32.238158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:32.238158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:32.238158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:32.238158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 246/498] 10.222.0.78 () {56 vars in 1102 bytes} [Wed May 25 14:41:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:32.238795 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c54407-169b-47e5-b09f-6017ec0e647e None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:32.243888 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68fadd-80e7-4b19-a18d-b420ff53ef8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:32.244551 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68fadd-80e7-4b19-a18d-b420ff53ef8a None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:32.244829 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68fadd-80e7-4b19-a18d-b420ff53ef8a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:32.245089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68fadd-80e7-4b19-a18d-b420ff53ef8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:32.325025 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68fadd-80e7-4b19-a18d-b420ff53ef8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:32.326533 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68fadd-80e7-4b19-a18d-b420ff53ef8a None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:32.356899 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c54407-169b-47e5-b09f-6017ec0e647e None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:32.393971 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4b039-46db-4d71-bf98-95c64ecb3b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:32.394678 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4b039-46db-4d71-bf98-95c64ecb3b80 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:32.394961 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4b039-46db-4d71-bf98-95c64ecb3b80 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:32.395067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:32.395235 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4b039-46db-4d71-bf98-95c64ecb3b80 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:32.395986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f4b039-46db-4d71-bf98-95c64ecb3b80 None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:32.396292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f4b039-46db-4d71-bf98-95c64ecb3b80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:32.397697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f4b039-46db-4d71-bf98-95c64ecb3b80 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:32.406204 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 247/499] 10.222.0.78 () {56 vars in 1103 bytes} [Wed May 25 14:41:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:32.407029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4b039-46db-4d71-bf98-95c64ecb3b80 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:32.443469 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:32.444122 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 253/500] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:41:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:41:32.444988 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68fadd-80e7-4b19-a18d-b420ff53ef8a None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:32.522485 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4b039-46db-4d71-bf98-95c64ecb3b80 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:32.553067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68fadd-80e7-4b19-a18d-b420ff53ef8a None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:32.560810 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274540f6-53a1-448c-a254-94fd3d51100c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:32.561630 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274540f6-53a1-448c-a254-94fd3d51100c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:32.561922 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274540f6-53a1-448c-a254-94fd3d51100c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:32.562558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274540f6-53a1-448c-a254-94fd3d51100c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:32.573895 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274540f6-53a1-448c-a254-94fd3d51100c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:32.575438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274540f6-53a1-448c-a254-94fd3d51100c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:32.588929 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72db7b-d67b-4f0c-87dd-6853a31495d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:32.589594 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72db7b-d67b-4f0c-87dd-6853a31495d0 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:32.589872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72db7b-d67b-4f0c-87dd-6853a31495d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:32.590175 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72db7b-d67b-4f0c-87dd-6853a31495d0 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:32.590926 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72db7b-d67b-4f0c-87dd-6853a31495d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:32.591230 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72db7b-d67b-4f0c-87dd-6853a31495d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:32.592603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72db7b-d67b-4f0c-87dd-6853a31495d0 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:32.593896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-274540f6-53a1-448c-a254-94fd3d51100c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:32.593896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:32.593896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:32.593896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:32.593896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:32.593896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:32.593896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:32.593896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:32.593896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:32.593896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:32.593896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:32.593896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:32.593896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:32.593896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:32.593896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:32.594869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:32.594869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:32.594869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:32.594869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:32.594869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:32.594869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:32.594869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:32.594869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:32.594869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:32.594869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:32.594869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:32.594869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:32.594869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:32.594869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:32.594869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:32.594869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:32.594869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:32.596085 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:32.596085 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:32.596085 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:32.596085 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:32.596085 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 248/501] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:32.596477 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d41c1b-df01-4914-8fb3-27524e6bdf3e None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:32.596780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d41c1b-df01-4914-8fb3-27524e6bdf3e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:32.596780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d41c1b-df01-4914-8fb3-27524e6bdf3e None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:32.598395 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 249/502] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:32.599120 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d41c1b-df01-4914-8fb3-27524e6bdf3e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:32.614560 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 254/503] 10.222.0.78 () {56 vars in 1103 bytes} [Wed May 25 14:41:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:32.615430 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72db7b-d67b-4f0c-87dd-6853a31495d0 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:32.643301 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:32.710991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:32.724679 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72db7b-d67b-4f0c-87dd-6853a31495d0 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:32.729261 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d41c1b-df01-4914-8fb3-27524e6bdf3e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:32.762709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d80f37-463c-4aec-858e-24674efe893a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:32.763910 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d80f37-463c-4aec-858e-24674efe893a None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:32.764365 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d80f37-463c-4aec-858e-24674efe893a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:32.764856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d80f37-463c-4aec-858e-24674efe893a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:32.782928 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d80f37-463c-4aec-858e-24674efe893a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:32.785247 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d80f37-463c-4aec-858e-24674efe893a None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:32.790971 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd0c9a-a28a-43a5-a141-b43d93eb29a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:32.792216 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd0c9a-a28a-43a5-a141-b43d93eb29a2 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:32.792681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd0c9a-a28a-43a5-a141-b43d93eb29a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:32.793149 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd0c9a-a28a-43a5-a141-b43d93eb29a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:32.810558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd0c9a-a28a-43a5-a141-b43d93eb29a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:32.813115 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd0c9a-a28a-43a5-a141-b43d93eb29a2 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:32.815362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-66d80f37-463c-4aec-858e-24674efe893a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:32.815362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:32.815362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:32.815362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:32.815362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:32.815362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:32.815362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:32.815362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:32.815362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:32.815362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:32.815362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:32.815362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:32.815362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:32.815362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:32.815362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:32.816204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:32.816204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:32.816204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:32.816204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:32.816204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:32.816204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:32.816204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:32.816204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:32.816204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:32.816204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:32.816204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:32.816204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:32.816204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:32.816204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:32.816204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:32.816204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:32.816204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:32.817641 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:32.817641 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:32.817641 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:32.817641 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:32.817641 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 255/504] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:32.818083 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d80f37-463c-4aec-858e-24674efe893a None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:32.844704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-91fd0c9a-a28a-43a5-a141-b43d93eb29a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:32.844704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:32.844704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:32.844704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:32.844704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:32.844704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:32.844704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:32.844704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:32.844704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:32.844704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:32.844704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:32.844704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:32.844704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:32.844704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:32.844704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:32.845616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:32.845616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:32.845616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:32.845616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:32.845616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:32.845616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:32.845616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:32.845616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:32.845616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:32.845616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:32.845616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:32.845616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:32.845616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:32.845616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:32.845616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:32.845616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:32.845616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:32.846502 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:32.846502 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:32.846502 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:32.846502 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:32.846969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 250/505] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:32.848468 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e183a-b1f8-4743-a350-1ad1b3f094ae None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:32.848854 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e183a-b1f8-4743-a350-1ad1b3f094ae None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:32.849225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e183a-b1f8-4743-a350-1ad1b3f094ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:32.868020 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-c45e183a-b1f8-4743-a350-1ad1b3f094ae None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:32.869461 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c45e183a-b1f8-4743-a350-1ad1b3f094ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:32.919596 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:32.949953 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d80f37-463c-4aec-858e-24674efe893a None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:32.952059 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 251/506] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:32.953476 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e183a-b1f8-4743-a350-1ad1b3f094ae None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:32.991645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd616a9-2d93-4e8d-8673-204bf86ebf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:32.992304 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd616a9-2d93-4e8d-8673-204bf86ebf17 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:32.992580 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd616a9-2d93-4e8d-8673-204bf86ebf17 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:32.992852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd616a9-2d93-4e8d-8673-204bf86ebf17 None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:32.993564 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd616a9-2d93-4e8d-8673-204bf86ebf17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:32.993865 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd616a9-2d93-4e8d-8673-204bf86ebf17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:32.995286 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd616a9-2d93-4e8d-8673-204bf86ebf17 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:33.009391 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 256/507] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:33.010263 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd616a9-2d93-4e8d-8673-204bf86ebf17 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:33.041991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:33.079956 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e183a-b1f8-4743-a350-1ad1b3f094ae None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:33.116874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44164ac-2c79-4aa6-9ee8-f99d45afd86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:33.117668 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44164ac-2c79-4aa6-9ee8-f99d45afd86d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:33.117949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44164ac-2c79-4aa6-9ee8-f99d45afd86d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:33.118251 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44164ac-2c79-4aa6-9ee8-f99d45afd86d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:33.122236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd616a9-2d93-4e8d-8673-204bf86ebf17 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:33.132293 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44164ac-2c79-4aa6-9ee8-f99d45afd86d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:33.133780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44164ac-2c79-4aa6-9ee8-f99d45afd86d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:33.152797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-d44164ac-2c79-4aa6-9ee8-f99d45afd86d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:33.152797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:33.152797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:33.152797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:33.152797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:33.152797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:33.152797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:33.152797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:33.152797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:33.152797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:33.152797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:33.152797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:33.152797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:33.152797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:33.152797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:33.154004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:33.154004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:33.154004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:33.154004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:33.154004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:33.154004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:33.154004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:33.154004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:33.154004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:33.154004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:33.154004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:33.154004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:33.154004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:33.154004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:33.154004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:33.154004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:33.154004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:33.154914 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:33.154914 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:33.154914 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:33.154914 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:33.154914 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 252/508] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:33.155177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44164ac-2c79-4aa6-9ee8-f99d45afd86d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:33.165372 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3a101-3f69-41c3-9185-3124bfeaaa04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:33.166075 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3a101-3f69-41c3-9185-3124bfeaaa04 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:33.166363 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3a101-3f69-41c3-9185-3124bfeaaa04 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:33.166639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3a101-3f69-41c3-9185-3124bfeaaa04 None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:33.167371 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae3a101-3f69-41c3-9185-3124bfeaaa04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:33.168129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae3a101-3f69-41c3-9185-3124bfeaaa04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:33.169514 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae3a101-3f69-41c3-9185-3124bfeaaa04 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:33.173280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:33.183336 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 257/509] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:33.183336 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3a101-3f69-41c3-9185-3124bfeaaa04 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:33.267233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44164ac-2c79-4aa6-9ee8-f99d45afd86d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:33.287907 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:33.297499 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3a101-3f69-41c3-9185-3124bfeaaa04 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:33.308358 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395964e-55b8-4a7b-93ad-be5dc31cefb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:33.309232 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395964e-55b8-4a7b-93ad-be5dc31cefb7 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:33.309716 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395964e-55b8-4a7b-93ad-be5dc31cefb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:33.310164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395964e-55b8-4a7b-93ad-be5dc31cefb7 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:33.310964 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395964e-55b8-4a7b-93ad-be5dc31cefb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:33.311407 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395964e-55b8-4a7b-93ad-be5dc31cefb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:33.312999 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395964e-55b8-4a7b-93ad-be5dc31cefb7 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:33.330438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 253/510] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:33.331787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395964e-55b8-4a7b-93ad-be5dc31cefb7 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:33.340604 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a67af-0225-49f2-97ec-b339e613dcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:33.341837 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a67af-0225-49f2-97ec-b339e613dcbe None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:33.342324 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a67af-0225-49f2-97ec-b339e613dcbe None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:33.342772 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a67af-0225-49f2-97ec-b339e613dcbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:33.361220 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a67af-0225-49f2-97ec-b339e613dcbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:33.363687 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a67af-0225-49f2-97ec-b339e613dcbe None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:33.393540 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-cc0a67af-0225-49f2-97ec-b339e613dcbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:33.393540 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:33.393540 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:33.393540 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:33.393540 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:33.393540 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:33.393540 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:33.393540 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:33.393540 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:33.393540 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:33.393540 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:33.393540 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:33.393540 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:33.393540 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:33.393540 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:33.394536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:33.394536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:33.394536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:33.394536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:33.394536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:33.394536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:33.394536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:33.394536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:33.394536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:33.394536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:33.394536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:33.394536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:33.394536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:33.394536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:33.394536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:33.394536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:33.394536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:33.395373 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:33.395373 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:33.395373 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:33.395373 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:33.395697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 258/511] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:33.397027 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a67af-0225-49f2-97ec-b339e613dcbe None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:33.453777 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:33.463585 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395964e-55b8-4a7b-93ad-be5dc31cefb7 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:33.503211 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de883a20-00db-4a34-b7ff-91185fcba106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:33.504012 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de883a20-00db-4a34-b7ff-91185fcba106 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:33.504295 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de883a20-00db-4a34-b7ff-91185fcba106 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:33.504622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de883a20-00db-4a34-b7ff-91185fcba106 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:33.511983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a67af-0225-49f2-97ec-b339e613dcbe None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:33.513030 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de883a20-00db-4a34-b7ff-91185fcba106 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:33.514530 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de883a20-00db-4a34-b7ff-91185fcba106 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:33.529424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-de883a20-00db-4a34-b7ff-91185fcba106 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 14:41:33.529424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:33.529424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:33.529424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:33.529424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:33.529424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:33.529424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:33.529424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:33.529424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:33.529424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:33.529424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:33.529424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:33.529424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:41:33.529424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:41:33.529424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:41:33.529424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:41:33.530720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:33.530720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:33.530720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:41:33.530720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:33.530720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:41:33.530720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:33.530720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:33.530720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:33.530720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:33.530720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:33.530720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:33.530720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:33.530720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:33.530720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:33.530720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:33.530720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:33.530720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:33.530720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:33.531929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:33.531929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:33.531929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:41:33.531929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:41:33.531929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:41:33.531929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:41:33.531929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:41:33.531929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:41:33.531929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 14:41:33.531929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:33.531929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 254/512] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:41:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:33.531929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de883a20-00db-4a34-b7ff-91185fcba106 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:33.551278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f90cc-c2a5-4e69-afd0-0fdafb05ff9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:33.554121 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f90cc-c2a5-4e69-afd0-0fdafb05ff9e None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:33.554121 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f90cc-c2a5-4e69-afd0-0fdafb05ff9e None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:33.554121 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f90cc-c2a5-4e69-afd0-0fdafb05ff9e None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:33.554121 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f90cc-c2a5-4e69-afd0-0fdafb05ff9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:33.554121 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f90cc-c2a5-4e69-afd0-0fdafb05ff9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:33.554935 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f90cc-c2a5-4e69-afd0-0fdafb05ff9e None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:33.568564 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 259/513] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:33.569494 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f90cc-c2a5-4e69-afd0-0fdafb05ff9e None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:33.581908 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:33.657031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de883a20-00db-4a34-b7ff-91185fcba106 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:33.683138 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f90cc-c2a5-4e69-afd0-0fdafb05ff9e None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:33.692581 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cccff8-8bb1-4d91-978c-93a321f2234d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:33.693386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cccff8-8bb1-4d91-978c-93a321f2234d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:33.693669 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cccff8-8bb1-4d91-978c-93a321f2234d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:33.693942 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cccff8-8bb1-4d91-978c-93a321f2234d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:33.697605 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:33.701884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cccff8-8bb1-4d91-978c-93a321f2234d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:33.703951 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cccff8-8bb1-4d91-978c-93a321f2234d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:33.720533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-c3cccff8-8bb1-4d91-978c-93a321f2234d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:33.720533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:33.720533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:33.720533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:33.720533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:33.720533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:33.720533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:33.720533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:33.720533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:33.720533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:33.720533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:33.720533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:33.720533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:41:33.720533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:41:33.720533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:41:33.721482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:41:33.721482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:33.721482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:33.721482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:33.721482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:33.721482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:33.721482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:33.721482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:33.721482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:33.721482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:33.721482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:33.721482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:33.721482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:33.721482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:33.721482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:33.721482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:33.721482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:33.722363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:41:33.722363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:41:33.722363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:41:33.722363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:41:33.722363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:41:33.722363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:41:33.722363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:33.722363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:33.722363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 255/514] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:41:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:33.722851 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cccff8-8bb1-4d91-978c-93a321f2234d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:33.731491 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2588a3ea-edda-446b-bfe2-503404caca0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:33.732288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588a3ea-edda-446b-bfe2-503404caca0d None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:33.733038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588a3ea-edda-446b-bfe2-503404caca0d None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:33.733306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588a3ea-edda-446b-bfe2-503404caca0d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:33.741154 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2588a3ea-edda-446b-bfe2-503404caca0d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:33.742650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2588a3ea-edda-446b-bfe2-503404caca0d None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:33.760579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-2588a3ea-edda-446b-bfe2-503404caca0d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 25 14:41:33.760579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:33.760579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:33.760579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:33.760579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:33.760579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:33.760579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:33.760579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:33.760579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:33.760579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:33.760579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:33.760579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:33.760579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:41:33.760579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:41:33.760579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:41:33.760579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:41:33.761530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:33.761530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:33.761530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:41:33.761530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:33.761530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:41:33.761530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:33.761530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:33.761530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:33.761530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:33.761530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:33.761530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:33.761530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:33.761530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:33.761530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:33.761530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:33.761530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:33.761530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:33.761530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:33.762447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:33.762447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:33.762447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:41:33.762447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:41:33.762447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:41:33.762447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:41:33.762447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:41:33.762447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:41:33.762447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 25 14:41:33.762447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:33.762447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 260/515] 10.222.0.78 () {56 vars in 1102 bytes} [Wed May 25 14:41:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:33.762993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2588a3ea-edda-446b-bfe2-503404caca0d None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:33.833251 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:33.839206 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cccff8-8bb1-4d91-978c-93a321f2234d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:33.878707 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c673f88-4e22-4935-adcc-f60d39671bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:33.879718 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c673f88-4e22-4935-adcc-f60d39671bdd None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:33.880229 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c673f88-4e22-4935-adcc-f60d39671bdd None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:33.880532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c673f88-4e22-4935-adcc-f60d39671bdd None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:33.881258 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c673f88-4e22-4935-adcc-f60d39671bdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:33.881568 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c673f88-4e22-4935-adcc-f60d39671bdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:33.883004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c673f88-4e22-4935-adcc-f60d39671bdd None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:33.883611 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2588a3ea-edda-446b-bfe2-503404caca0d None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:33.897192 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 256/516] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:33.898082 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c673f88-4e22-4935-adcc-f60d39671bdd None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:33.926666 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99816b2-85a1-4cd4-9a3f-c487bc07c819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:33.927352 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99816b2-85a1-4cd4-9a3f-c487bc07c819 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:33.927631 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99816b2-85a1-4cd4-9a3f-c487bc07c819 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:33.927905 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99816b2-85a1-4cd4-9a3f-c487bc07c819 None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:33.928746 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99816b2-85a1-4cd4-9a3f-c487bc07c819 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:33.929053 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99816b2-85a1-4cd4-9a3f-c487bc07c819 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:33.930547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99816b2-85a1-4cd4-9a3f-c487bc07c819 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:33.953874 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 261/517] 10.222.0.78 () {56 vars in 1137 bytes} [Wed May 25 14:41:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:33.953874 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99816b2-85a1-4cd4-9a3f-c487bc07c819 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:33.981807 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:34.007941 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c673f88-4e22-4935-adcc-f60d39671bdd None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:34.049544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a7adf8-e0f6-4684-b561-c2d124b37753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:34.050363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7adf8-e0f6-4684-b561-c2d124b37753 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:34.050643 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7adf8-e0f6-4684-b561-c2d124b37753 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:34.050912 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7adf8-e0f6-4684-b561-c2d124b37753 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:34.058262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a7adf8-e0f6-4684-b561-c2d124b37753 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:34.059729 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a7adf8-e0f6-4684-b561-c2d124b37753 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:34.066260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99816b2-85a1-4cd4-9a3f-c487bc07c819 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:34.075140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-37a7adf8-e0f6-4684-b561-c2d124b37753 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 25 14:41:34.075140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:34.075140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:34.075140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:34.075140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:34.075140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:34.075140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:34.075140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:34.075140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:34.075140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:34.075140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:34.075140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:34.075140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:41:34.075140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:41:34.075140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:41:34.075140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:41:34.076055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:34.076055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:34.076055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:41:34.076055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:34.076055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:41:34.076055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:34.076055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:34.076055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:34.076055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:34.076055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:34.076055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:34.076055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:34.076055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:34.076055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:34.076055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:34.076055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:34.076055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:34.076055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:34.076944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:34.076944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:34.076944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:41:34.076944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:41:34.076944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:41:34.076944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:41:34.076944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:41:34.076944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:41:34.076944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 25 14:41:34.076944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:34.076944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 257/518] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:41:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:34.077552 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a7adf8-e0f6-4684-b561-c2d124b37753 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:34.101453 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:34.106188 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183de00c-5e65-4374-a2e9-ac0d880ac0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:34.106857 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183de00c-5e65-4374-a2e9-ac0d880ac0bc None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:34.107136 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183de00c-5e65-4374-a2e9-ac0d880ac0bc None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:34.107407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183de00c-5e65-4374-a2e9-ac0d880ac0bc None admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:34.108189 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183de00c-5e65-4374-a2e9-ac0d880ac0bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:34.108492 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183de00c-5e65-4374-a2e9-ac0d880ac0bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:34.110552 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183de00c-5e65-4374-a2e9-ac0d880ac0bc None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:34.129173 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 262/519] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:41:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:34.129823 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183de00c-5e65-4374-a2e9-ac0d880ac0bc None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:34.189307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a7adf8-e0f6-4684-b561-c2d124b37753 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:34.224660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:34.226582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdac47-bca5-4368-933c-8e2538623c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:34.227340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdac47-bca5-4368-933c-8e2538623c7b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:34.227698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdac47-bca5-4368-933c-8e2538623c7b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:34.228041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdac47-bca5-4368-933c-8e2538623c7b None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:34.228957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fdac47-bca5-4368-933c-8e2538623c7b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:34.229342 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fdac47-bca5-4368-933c-8e2538623c7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:34.230830 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fdac47-bca5-4368-933c-8e2538623c7b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:34.242136 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183de00c-5e65-4374-a2e9-ac0d880ac0bc None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:34.274881 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 258/520] 10.222.0.78 () {56 vars in 1139 bytes} [Wed May 25 14:41:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:34.275772 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdac47-bca5-4368-933c-8e2538623c7b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:34.289556 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef13af-f4f7-4f68-acdc-20b27e83c380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:34.290281 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef13af-f4f7-4f68-acdc-20b27e83c380 None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:34.290551 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef13af-f4f7-4f68-acdc-20b27e83c380 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:34.290822 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef13af-f4f7-4f68-acdc-20b27e83c380 None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:34.291532 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef13af-f4f7-4f68-acdc-20b27e83c380 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:34.292991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef13af-f4f7-4f68-acdc-20b27e83c380 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:34.316689 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 263/521] 10.222.0.78 () {60 vars in 1132 bytes} [Wed May 25 14:41:34 2022] POST /identity/v3/users => generated 247 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:34.317350 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef13af-f4f7-4f68-acdc-20b27e83c380 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:34.394439 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:34.428735 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdac47-bca5-4368-933c-8e2538623c7b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:34.447395 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef13af-f4f7-4f68-acdc-20b27e83c380 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:34.471104 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171adf7c-ac94-4b4c-b6e6-bde20633c8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:34.471787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171adf7c-ac94-4b4c-b6e6-bde20633c8d7 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:34.472078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171adf7c-ac94-4b4c-b6e6-bde20633c8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:34.472358 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171adf7c-ac94-4b4c-b6e6-bde20633c8d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:34.490680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c1713-b972-40bd-a2ea-354b275e0d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:34.490680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c1713-b972-40bd-a2ea-354b275e0d48 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:34.490680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c1713-b972-40bd-a2ea-354b275e0d48 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:34.490680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c1713-b972-40bd-a2ea-354b275e0d48 None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:34.492945 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c1713-b972-40bd-a2ea-354b275e0d48 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:34.493292 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c1713-b972-40bd-a2ea-354b275e0d48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:34.494655 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c1713-b972-40bd-a2ea-354b275e0d48 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:34.511876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:34.520752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 264/522] 10.222.0.78 () {56 vars in 1137 bytes} [Wed May 25 14:41:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:34.522978 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c1713-b972-40bd-a2ea-354b275e0d48 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:34.571236 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171adf7c-ac94-4b4c-b6e6-bde20633c8d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:34.572734 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171adf7c-ac94-4b4c-b6e6-bde20633c8d7 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:34.655194 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c1713-b972-40bd-a2ea-354b275e0d48 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:34.669586 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:34.699574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 259/523] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:41:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:41:34.700601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171adf7c-ac94-4b4c-b6e6-bde20633c8d7 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:34.706075 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366113b3-296f-496e-b5ee-d0e57835ce3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:34.706843 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366113b3-296f-496e-b5ee-d0e57835ce3d None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:34.707120 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366113b3-296f-496e-b5ee-d0e57835ce3d None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:34.707400 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366113b3-296f-496e-b5ee-d0e57835ce3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:34.708264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366113b3-296f-496e-b5ee-d0e57835ce3d None admin] RBAC: Adding query filter params (role.id=336f27e7f8624a30b30cef9acada97a0, scope.project.id=908a13d2b49544fbaf760b07e213bb7d, user.id=e34e1d7ffbac49b0bde45829af0c6925) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:34.708577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366113b3-296f-496e-b5ee-d0e57835ce3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:34.710115 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366113b3-296f-496e-b5ee-d0e57835ce3d None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:34.726059 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 265/524] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:41:34 2022] GET /identity/v3/role_assignments?user.id=e34e1d7ffbac49b0bde45829af0c6925&scope.project.id=908a13d2b49544fbaf760b07e213bb7d&role.id=336f27e7f8624a30b30cef9acada97a0 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:34.726059 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366113b3-296f-496e-b5ee-d0e57835ce3d None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:34.824911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:34.876533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171adf7c-ac94-4b4c-b6e6-bde20633c8d7 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:34.882293 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366113b3-296f-496e-b5ee-d0e57835ce3d None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:34.934357 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e6cf4-f647-4804-baac-3bc5c9c2c3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:34.935725 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e6cf4-f647-4804-baac-3bc5c9c2c3f7 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:34.936008 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e6cf4-f647-4804-baac-3bc5c9c2c3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:34.936290 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e6cf4-f647-4804-baac-3bc5c9c2c3f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:34.942862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c579012-01f3-47fd-97d5-e83783a23303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:34.943855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c579012-01f3-47fd-97d5-e83783a23303 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:34.944373 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c579012-01f3-47fd-97d5-e83783a23303 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:34.944964 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c579012-01f3-47fd-97d5-e83783a23303 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:35.011865 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:35.029787 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086e6cf4-f647-4804-baac-3bc5c9c2c3f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:35.032188 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086e6cf4-f647-4804-baac-3bc5c9c2c3f7 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:35.048087 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c579012-01f3-47fd-97d5-e83783a23303 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:35.049964 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c579012-01f3-47fd-97d5-e83783a23303 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:35.122448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:35.183319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 266/525] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:41:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:41:35.183319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e6cf4-f647-4804-baac-3bc5c9c2c3f7 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:35.184072 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 260/526] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:41:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:41:35.185331 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c579012-01f3-47fd-97d5-e83783a23303 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:35.305545 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:35.320843 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e6cf4-f647-4804-baac-3bc5c9c2c3f7 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:35.338754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c579012-01f3-47fd-97d5-e83783a23303 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:35.377678 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0cdd57-6190-4658-9c99-5187be4d6de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:35.378753 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2477e-68ed-4e53-ada2-de3a0fbce1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:35.379231 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cdd57-6190-4658-9c99-5187be4d6de3 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:35.379543 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2477e-68ed-4e53-ada2-de3a0fbce1cf None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:35.379826 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2477e-68ed-4e53-ada2-de3a0fbce1cf None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:35.380100 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2477e-68ed-4e53-ada2-de3a0fbce1cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:35.380462 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cdd57-6190-4658-9c99-5187be4d6de3 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:35.381016 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cdd57-6190-4658-9c99-5187be4d6de3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:35.390960 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb2477e-68ed-4e53-ada2-de3a0fbce1cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:35.392418 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb2477e-68ed-4e53-ada2-de3a0fbce1cf None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:35.394667 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0cdd57-6190-4658-9c99-5187be4d6de3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:35.396504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0cdd57-6190-4658-9c99-5187be4d6de3 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:35.411972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-feb2477e-68ed-4e53-ada2-de3a0fbce1cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:35.411972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:35.411972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:35.411972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:35.411972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:35.411972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:35.411972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:35.411972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:35.411972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:35.411972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:35.411972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:35.411972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:35.411972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:35.411972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:35.411972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:35.412877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:35.412877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:35.412877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:35.412877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:35.412877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:35.412877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:35.412877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:35.412877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:35.412877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:35.412877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:35.412877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:35.412877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:35.412877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:35.412877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:35.412877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:35.412877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:35.412877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:35.413720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:35.413720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:35.413720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:35.413720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:35.413720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 261/527] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:35.414426 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2477e-68ed-4e53-ada2-de3a0fbce1cf None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:35.420130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-ef0cdd57-6190-4658-9c99-5187be4d6de3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:35.420130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:35.420130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:35.420130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:35.420130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:35.420130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:35.420130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:35.420130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:35.420130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:35.420130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:35.420130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:35.420130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:35.420130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:35.420130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:35.420130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:35.421009 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:35.421009 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:35.421009 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:35.421009 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:35.421009 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:35.421009 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:35.421009 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:35.421009 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:35.421009 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:35.421009 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:35.421009 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:35.421009 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:35.421009 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:35.421009 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:35.421009 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:35.421009 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:35.421009 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:35.422830 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:35.422830 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:35.422830 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:35.422830 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:35.425014 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 267/528] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:35.426165 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0cdd57-6190-4658-9c99-5187be4d6de3 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:35.449825 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:35.538716 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2477e-68ed-4e53-ada2-de3a0fbce1cf None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:35.552717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0cdd57-6190-4658-9c99-5187be4d6de3 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:35.568416 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:35.579695 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d365fc-5935-4231-87e9-726266106c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:35.580526 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d365fc-5935-4231-87e9-726266106c2d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:35.580774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d365fc-5935-4231-87e9-726266106c2d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:35.581047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d365fc-5935-4231-87e9-726266106c2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:35.604266 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d365fc-5935-4231-87e9-726266106c2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:35.605771 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d365fc-5935-4231-87e9-726266106c2d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:35.609545 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7e29d-bd8f-4edc-97ad-0b83f6775127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:35.610431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7e29d-bd8f-4edc-97ad-0b83f6775127 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:35.610946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7e29d-bd8f-4edc-97ad-0b83f6775127 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:35.611224 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7e29d-bd8f-4edc-97ad-0b83f6775127 None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:35.612181 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde7e29d-bd8f-4edc-97ad-0b83f6775127 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:35.612491 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde7e29d-bd8f-4edc-97ad-0b83f6775127 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:35.614754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde7e29d-bd8f-4edc-97ad-0b83f6775127 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:35.627307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-a0d365fc-5935-4231-87e9-726266106c2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:35.627307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:35.627307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:35.627307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:35.627307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:35.627307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:35.627307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:35.627307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:35.627307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:35.627307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:35.627307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:35.627307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:35.627307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:35.627307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:35.627307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:35.628228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:35.628228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:35.628228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:35.628228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:35.628228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:35.628228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:35.628228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:35.628228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:35.628228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:35.628228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:35.628228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:35.628228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:35.628228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:35.628228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:35.628228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:35.628228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:35.628228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:35.629055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:35.629055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:35.629055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:35.629055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:35.631106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 262/529] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:35.631106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d365fc-5935-4231-87e9-726266106c2d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:35.636928 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 268/530] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:35.642233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7e29d-bd8f-4edc-97ad-0b83f6775127 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:35.725872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:35.742461 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d365fc-5935-4231-87e9-726266106c2d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:35.753193 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7e29d-bd8f-4edc-97ad-0b83f6775127 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:35.778424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5262de-a08f-4d7d-a58d-37f84dff9f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:35.779122 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5262de-a08f-4d7d-a58d-37f84dff9f58 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:35.779412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5262de-a08f-4d7d-a58d-37f84dff9f58 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:35.779753 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5262de-a08f-4d7d-a58d-37f84dff9f58 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:35.780501 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5262de-a08f-4d7d-a58d-37f84dff9f58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:35.781404 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5262de-a08f-4d7d-a58d-37f84dff9f58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:35.782935 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5262de-a08f-4d7d-a58d-37f84dff9f58 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:35.797257 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734e857-ea6d-44c7-90aa-080d8d8c6e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:35.798271 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734e857-ea6d-44c7-90aa-080d8d8c6e10 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:35.798797 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734e857-ea6d-44c7-90aa-080d8d8c6e10 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:35.799241 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734e857-ea6d-44c7-90aa-080d8d8c6e10 None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:35.800144 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734e857-ea6d-44c7-90aa-080d8d8c6e10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:35.800641 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734e857-ea6d-44c7-90aa-080d8d8c6e10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:35.802297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 263/531] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:35.802627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734e857-ea6d-44c7-90aa-080d8d8c6e10 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:35.803328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5262de-a08f-4d7d-a58d-37f84dff9f58 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:35.818979 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 269/532] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:35.820221 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734e857-ea6d-44c7-90aa-080d8d8c6e10 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:35.874539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:35.954169 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5262de-a08f-4d7d-a58d-37f84dff9f58 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:35.973487 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734e857-ea6d-44c7-90aa-080d8d8c6e10 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:36.006899 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744722b-14ca-479e-b273-5d80ab491b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:36.007942 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744722b-14ca-479e-b273-5d80ab491b14 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:36.008495 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744722b-14ca-479e-b273-5d80ab491b14 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:36.009055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744722b-14ca-479e-b273-5d80ab491b14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:36.017918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4744722b-14ca-479e-b273-5d80ab491b14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:36.019634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4744722b-14ca-479e-b273-5d80ab491b14 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:36.024237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b15719-1915-407f-b070-1e876928844a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:36.025226 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b15719-1915-407f-b070-1e876928844a None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:36.025653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b15719-1915-407f-b070-1e876928844a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:36.026099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b15719-1915-407f-b070-1e876928844a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:36.029632 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:36.035546 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b15719-1915-407f-b070-1e876928844a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:36.038240 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b15719-1915-407f-b070-1e876928844a None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:36.039564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-4744722b-14ca-479e-b273-5d80ab491b14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:36.039564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:36.039564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:36.039564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:36.039564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:36.039564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:36.039564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:36.039564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:36.039564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:36.039564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:36.039564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:36.039564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:36.039564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:41:36.039564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:41:36.039564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:41:36.040393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:41:36.040393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:36.040393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:36.040393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:36.040393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:36.040393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:36.040393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:36.040393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:36.040393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:36.040393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:36.040393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:36.040393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:36.040393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:36.040393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:36.040393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:36.040393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:36.040393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:36.041234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:41:36.041234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:41:36.041234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:41:36.041234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:41:36.041234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:41:36.041234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:41:36.041234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:36.041234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:36.043424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 264/533] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:41:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:36.044471 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744722b-14ca-479e-b273-5d80ab491b14 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:36.063158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-33b15719-1915-407f-b070-1e876928844a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:36.063158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:36.063158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:36.063158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:36.063158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:36.063158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:36.063158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:36.063158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:36.063158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:36.063158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:36.063158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:36.063158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:36.063158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:41:36.063158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:41:36.063158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:41:36.064454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:41:36.064454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:36.064454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:36.064454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:36.064454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:36.064454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:36.064454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:36.064454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:36.064454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:36.064454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:36.064454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:36.064454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:36.064454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:36.064454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:36.064454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:36.064454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:36.064454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:36.065386 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:41:36.065386 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:41:36.065386 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:41:36.065386 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:41:36.065386 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:41:36.065386 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:41:36.065386 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:36.065386 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:36.069380 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 270/534] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:41:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:36.070328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b15719-1915-407f-b070-1e876928844a None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:36.168084 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:36.172945 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744722b-14ca-479e-b273-5d80ab491b14 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:36.212114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b15719-1915-407f-b070-1e876928844a None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:36.215329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221627b-a13d-4a23-8f1a-f6e6d47faa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:36.216403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221627b-a13d-4a23-8f1a-f6e6d47faa4f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:36.216935 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221627b-a13d-4a23-8f1a-f6e6d47faa4f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:36.217455 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221627b-a13d-4a23-8f1a-f6e6d47faa4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:36.226354 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221627b-a13d-4a23-8f1a-f6e6d47faa4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:36.228399 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221627b-a13d-4a23-8f1a-f6e6d47faa4f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:36.246638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-6221627b-a13d-4a23-8f1a-f6e6d47faa4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:36.246638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:36.246638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:36.246638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:36.246638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:36.246638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:36.246638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:36.246638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:36.246638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:36.246638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:36.246638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:36.246638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:36.246638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:41:36.246638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:41:36.246638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:41:36.247554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:41:36.247554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:36.247554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:36.247554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:36.247554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:36.247554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:36.247554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:36.247554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:36.247554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:36.247554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:36.247554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:36.247554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:36.247554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:36.247554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:36.247554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:36.247554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:36.247554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:36.248390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:41:36.248390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:41:36.248390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:41:36.248390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:41:36.248390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:41:36.248390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:41:36.248390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:36.248390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:36.254159 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 265/535] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:41:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:36.255621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221627b-a13d-4a23-8f1a-f6e6d47faa4f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:36.257835 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43faa6-2c22-4390-9941-956c46600923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:36.258545 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43faa6-2c22-4390-9941-956c46600923 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:36.258829 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43faa6-2c22-4390-9941-956c46600923 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:36.259104 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43faa6-2c22-4390-9941-956c46600923 None admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:36.260303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43faa6-2c22-4390-9941-956c46600923 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:36.260620 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43faa6-2c22-4390-9941-956c46600923 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:36.263063 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43faa6-2c22-4390-9941-956c46600923 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:36.279627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 271/536] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:41:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:36.281492 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43faa6-2c22-4390-9941-956c46600923 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:36.361560 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:36.399110 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221627b-a13d-4a23-8f1a-f6e6d47faa4f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:36.421311 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43faa6-2c22-4390-9941-956c46600923 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:36.453921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ddd90-5852-4a1a-b219-c3adc39e2b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:36.455532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ddd90-5852-4a1a-b219-c3adc39e2b51 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:36.456265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ddd90-5852-4a1a-b219-c3adc39e2b51 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:36.456965 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ddd90-5852-4a1a-b219-c3adc39e2b51 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:36.458686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ddd90-5852-4a1a-b219-c3adc39e2b51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:36.459450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ddd90-5852-4a1a-b219-c3adc39e2b51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:36.462295 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ddd90-5852-4a1a-b219-c3adc39e2b51 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:36.465985 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d6c30-f27a-4f42-b6f0-03c1b007a2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:36.466675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d6c30-f27a-4f42-b6f0-03c1b007a2f0 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:36.466961 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d6c30-f27a-4f42-b6f0-03c1b007a2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:36.467236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d6c30-f27a-4f42-b6f0-03c1b007a2f0 None admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:36.468581 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d6c30-f27a-4f42-b6f0-03c1b007a2f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:36.468894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d6c30-f27a-4f42-b6f0-03c1b007a2f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:36.470383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d6c30-f27a-4f42-b6f0-03c1b007a2f0 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:36.490437 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 272/537] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:41:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:36.491711 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d6c30-f27a-4f42-b6f0-03c1b007a2f0 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:36.496168 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 266/538] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:41:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:36.497372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ddd90-5852-4a1a-b219-c3adc39e2b51 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:36.524887 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:36.613628 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d6c30-f27a-4f42-b6f0-03c1b007a2f0 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:36.631301 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:36.648570 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ddd90-5852-4a1a-b219-c3adc39e2b51 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:36.651119 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5e681-a05e-481f-8045-a9609392a036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:36.652591 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5e681-a05e-481f-8045-a9609392a036 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:36.652591 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5e681-a05e-481f-8045-a9609392a036 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:36.652591 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5e681-a05e-481f-8045-a9609392a036 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:36.653149 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d5e681-a05e-481f-8045-a9609392a036 None admin] RBAC: Adding query filter params (role.id=336f27e7f8624a30b30cef9acada97a0, scope.project.id=908a13d2b49544fbaf760b07e213bb7d, user.id=88cbd76f70d447c4ba92777b2c09821d) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:36.653457 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d5e681-a05e-481f-8045-a9609392a036 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:36.654976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d5e681-a05e-481f-8045-a9609392a036 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:36.662268 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 273/539] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:41:36 2022] GET /identity/v3/role_assignments?user.id=88cbd76f70d447c4ba92777b2c09821d&scope.project.id=908a13d2b49544fbaf760b07e213bb7d&role.id=336f27e7f8624a30b30cef9acada97a0 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:36.662948 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5e681-a05e-481f-8045-a9609392a036 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:36.692811 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c11a74-c525-462e-878a-fab02af88510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:36.694038 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c11a74-c525-462e-878a-fab02af88510 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:36.694545 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c11a74-c525-462e-878a-fab02af88510 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:36.695109 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c11a74-c525-462e-878a-fab02af88510 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:36.696221 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c11a74-c525-462e-878a-fab02af88510 None admin] RBAC: Adding query filter params (role.id=336f27e7f8624a30b30cef9acada97a0, scope.project.id=908a13d2b49544fbaf760b07e213bb7d, user.id=70ad51a8ad68440da6618891e2e08cb8) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:36.696747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c11a74-c525-462e-878a-fab02af88510 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:36.698590 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c11a74-c525-462e-878a-fab02af88510 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:36.706712 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 267/540] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:41:36 2022] GET /identity/v3/role_assignments?user.id=70ad51a8ad68440da6618891e2e08cb8&scope.project.id=908a13d2b49544fbaf760b07e213bb7d&role.id=336f27e7f8624a30b30cef9acada97a0 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:36.764287 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3df7d-77fc-404d-8733-2b0d00def564 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:36.764649 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3df7d-77fc-404d-8733-2b0d00def564 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:36.764889 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3df7d-77fc-404d-8733-2b0d00def564 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:36.766794 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 268/541] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:41:36.773280 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350f669-e3a0-4cff-9560-f2c7d43e7609 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:36.773598 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350f669-e3a0-4cff-9560-f2c7d43e7609 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:36.773849 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350f669-e3a0-4cff-9560-f2c7d43e7609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:36.815321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-5350f669-e3a0-4cff-9560-f2c7d43e7609 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:36.819737 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5350f669-e3a0-4cff-9560-f2c7d43e7609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:36.893000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5e681-a05e-481f-8045-a9609392a036 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:36.964474 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 269/542] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:36.990943 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350f669-e3a0-4cff-9560-f2c7d43e7609 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:37.008479 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b18da2-faf4-4f9d-9bfa-8cb33feab400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:37.009153 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b18da2-faf4-4f9d-9bfa-8cb33feab400 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:37.009437 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b18da2-faf4-4f9d-9bfa-8cb33feab400 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:37.009712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b18da2-faf4-4f9d-9bfa-8cb33feab400 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:37.010725 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b18da2-faf4-4f9d-9bfa-8cb33feab400 None admin] RBAC: Adding query filter params (role.id=336f27e7f8624a30b30cef9acada97a0, scope.project.id=908a13d2b49544fbaf760b07e213bb7d, user.id=50dbd3dd7e87481cb940a467c294006c) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:37.010917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:37.011071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b18da2-faf4-4f9d-9bfa-8cb33feab400 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:37.012866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b18da2-faf4-4f9d-9bfa-8cb33feab400 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:37.020757 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 274/543] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:41:36 2022] GET /identity/v3/role_assignments?user.id=50dbd3dd7e87481cb940a467c294006c&scope.project.id=908a13d2b49544fbaf760b07e213bb7d&role.id=336f27e7f8624a30b30cef9acada97a0 => generated 259 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:37.133632 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca93c7-b35d-4d0f-99f6-ee59bb5cbb3b None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:37.134338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca93c7-b35d-4d0f-99f6-ee59bb5cbb3b None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:37.134338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca93c7-b35d-4d0f-99f6-ee59bb5cbb3b None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:37.135876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 275/544] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:41:37.142063 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693df04-ceed-4ee6-99a7-562a161a050f None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:37.142063 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693df04-ceed-4ee6-99a7-562a161a050f None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:37.142063 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693df04-ceed-4ee6-99a7-562a161a050f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:37.149064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350f669-e3a0-4cff-9560-f2c7d43e7609 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:37.195198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-8693df04-ceed-4ee6-99a7-562a161a050f None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:37.196350 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e3797-e5db-4ad2-9e19-940e5e3cd237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:37.196816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-8693df04-ceed-4ee6-99a7-562a161a050f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:37.197929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e3797-e5db-4ad2-9e19-940e5e3cd237 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:37.198447 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e3797-e5db-4ad2-9e19-940e5e3cd237 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:37.199242 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e3797-e5db-4ad2-9e19-940e5e3cd237 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:37.214314 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e3797-e5db-4ad2-9e19-940e5e3cd237 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:37.224553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-128e3797-e5db-4ad2-9e19-940e5e3cd237 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:37.224553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:37.224553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:37.224553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:37.224553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:37.224553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:37.224553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:37.224553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:37.224553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:37.224553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:37.224553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:37.224553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:37.224553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:37.224553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:37.224553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:37.224553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:37.225518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:37.225518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:37.225518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:37.225518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:37.225518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:37.225518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:37.225518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:37.225518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:37.225518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:37.225518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:37.225518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:37.225518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:37.225518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:37.225518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:37.225518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:37.225518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:37.226357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:37.226357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:37.226357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:37.226357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:37.226357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:37.226357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:37.226357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:37.226357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:37.226357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:37.226357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:37.226357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:37.226357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:37.229254 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 270/545] 10.222.0.78 () {56 vars in 1102 bytes} [Wed May 25 14:41:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:41:37.237009 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e3797-e5db-4ad2-9e19-940e5e3cd237 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:37.318551 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:37.370103 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:37.402610 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e3797-e5db-4ad2-9e19-940e5e3cd237 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:37.420499 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 276/546] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:41:37.441752 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a05e7-da69-4313-8553-7469e43e066f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:37.442751 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a05e7-da69-4313-8553-7469e43e066f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:37.443179 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a05e7-da69-4313-8553-7469e43e066f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:37.443594 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a05e7-da69-4313-8553-7469e43e066f None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:37.444511 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a05e7-da69-4313-8553-7469e43e066f None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:37.445012 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a05e7-da69-4313-8553-7469e43e066f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:37.446633 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a05e7-da69-4313-8553-7469e43e066f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:37.457201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 271/547] 10.222.0.78 () {56 vars in 1103 bytes} [Wed May 25 14:41:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:37.463497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693df04-ceed-4ee6-99a7-562a161a050f None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:37.469677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a05e7-da69-4313-8553-7469e43e066f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:37.592155 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a05e7-da69-4313-8553-7469e43e066f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:37.632973 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2d06a-cfb2-44e4-aeba-ad8fc288f862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:37.634296 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2d06a-cfb2-44e4-aeba-ad8fc288f862 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:37.634727 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2d06a-cfb2-44e4-aeba-ad8fc288f862 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:37.635172 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2d06a-cfb2-44e4-aeba-ad8fc288f862 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:37.638971 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:37.661445 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2d06a-cfb2-44e4-aeba-ad8fc288f862 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:37.678041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-43e2d06a-cfb2-44e4-aeba-ad8fc288f862 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:37.678041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:37.678041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:37.678041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:37.678041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:37.678041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:37.678041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:37.678041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:37.678041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:37.678041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:37.678041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:37.678041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:37.678041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:37.678041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:37.678041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:37.678041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:37.679157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:37.679157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:37.679157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:37.679157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:37.679157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:37.679157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:37.679157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:37.679157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:37.679157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:37.679157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:37.679157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:37.679157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:37.679157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:37.679157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:37.679157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:37.679157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:37.680051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:37.680051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:37.680051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:37.680051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:37.680051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:37.680051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:37.680051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:37.680051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:37.680051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:37.680051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:37.680051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:37.680051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:37.682995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 272/548] 10.222.0.78 () {56 vars in 1102 bytes} [Wed May 25 14:41:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:41:37.689308 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2d06a-cfb2-44e4-aeba-ad8fc288f862 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:37.704014 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693df04-ceed-4ee6-99a7-562a161a050f None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:37.744546 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbc2aa4-cdba-4b88-a7ff-65060431eace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:37.745499 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc2aa4-cdba-4b88-a7ff-65060431eace None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:37.745894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc2aa4-cdba-4b88-a7ff-65060431eace None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:37.746299 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc2aa4-cdba-4b88-a7ff-65060431eace None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:37.767897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbc2aa4-cdba-4b88-a7ff-65060431eace None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:37.769375 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbc2aa4-cdba-4b88-a7ff-65060431eace None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:37.797759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-dbbc2aa4-cdba-4b88-a7ff-65060431eace None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:37.797759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:37.797759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:37.797759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:37.797759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:37.797759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:37.797759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:37.797759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:37.797759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:37.797759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:37.797759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:37.797759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:37.797759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:37.797759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:37.797759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:37.798884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:37.798884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:37.798884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:37.798884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:37.798884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:37.798884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:37.798884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:37.798884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:37.798884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:37.798884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:37.798884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:37.798884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:37.798884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:37.798884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:37.798884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:37.798884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:37.798884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:37.799734 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:37.799734 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:37.799734 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:37.799734 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:37.799734 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 277/549] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:37.805533 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbc2aa4-cdba-4b88-a7ff-65060431eace None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:37.865843 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:37.936095 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2d06a-cfb2-44e4-aeba-ad8fc288f862 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:37.951336 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbc2aa4-cdba-4b88-a7ff-65060431eace None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:38.004272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09d97f-7560-4508-a466-45d0ae1297d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:38.005023 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09d97f-7560-4508-a466-45d0ae1297d7 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:38.005311 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09d97f-7560-4508-a466-45d0ae1297d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:38.005590 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09d97f-7560-4508-a466-45d0ae1297d7 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:38.006481 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09d97f-7560-4508-a466-45d0ae1297d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:38.007019 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09d97f-7560-4508-a466-45d0ae1297d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:38.008400 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09d97f-7560-4508-a466-45d0ae1297d7 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:38.015279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f19839-5149-4f61-81ca-93d91f937e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:38.015976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f19839-5149-4f61-81ca-93d91f937e34 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:38.016251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f19839-5149-4f61-81ca-93d91f937e34 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:38.016519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f19839-5149-4f61-81ca-93d91f937e34 None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:38.017250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f19839-5149-4f61-81ca-93d91f937e34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:38.017550 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f19839-5149-4f61-81ca-93d91f937e34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:38.018793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 273/550] 10.222.0.78 () {56 vars in 1103 bytes} [Wed May 25 14:41:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:38.018968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f19839-5149-4f61-81ca-93d91f937e34 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:38.024433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09d97f-7560-4508-a466-45d0ae1297d7 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:38.044530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 278/551] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:38.050197 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f19839-5149-4f61-81ca-93d91f937e34 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:38.156754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:38.212859 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f19839-5149-4f61-81ca-93d91f937e34 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:38.214407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:38.216680 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09d97f-7560-4508-a466-45d0ae1297d7 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:38.261951 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25f694-66fb-4011-8b1e-b2a814e0a888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:38.263657 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25f694-66fb-4011-8b1e-b2a814e0a888 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:38.264563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14043294-0fb9-4fd7-8411-23a0317a479c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:38.265618 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14043294-0fb9-4fd7-8411-23a0317a479c None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:38.266175 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25f694-66fb-4011-8b1e-b2a814e0a888 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:38.266654 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25f694-66fb-4011-8b1e-b2a814e0a888 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:38.267754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14043294-0fb9-4fd7-8411-23a0317a479c None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:38.268205 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14043294-0fb9-4fd7-8411-23a0317a479c None admin] PATH_INFO: `/v3/users/swift` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:38.283415 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de25f694-66fb-4011-8b1e-b2a814e0a888 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:38.285338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de25f694-66fb-4011-8b1e-b2a814e0a888 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:38.299282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14043294-0fb9-4fd7-8411-23a0317a479c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:38.301104 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14043294-0fb9-4fd7-8411-23a0317a479c None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:38.314472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-de25f694-66fb-4011-8b1e-b2a814e0a888 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:38.314472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:38.314472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:38.314472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:38.314472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:38.314472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:38.314472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:38.314472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:38.314472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:38.314472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:38.314472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:38.314472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:38.314472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:38.314472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:38.314472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:38.315386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:38.315386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:38.315386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:38.315386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:38.315386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:38.315386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:38.315386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:38.315386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:38.315386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:38.315386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:38.315386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:38.315386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:38.315386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:38.315386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:38.315386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:38.315386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:38.315386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:38.316252 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:38.316252 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:38.316252 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:38.316252 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:38.318290 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 274/552] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:38.321696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-14043294-0fb9-4fd7-8411-23a0317a479c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 14:41:38.321696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:38.321696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:38.321696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:38.321696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:38.321696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:38.321696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:38.321696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:38.321696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:38.321696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:38.321696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:38.321696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:38.321696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:41:38.321696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:41:38.321696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:41:38.321696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:41:38.323031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:38.323031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:38.323031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:41:38.323031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:38.323031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:41:38.323031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:38.323031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:38.323031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:38.323031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:38.323031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:38.323031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:38.323031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:38.323031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:38.323031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:38.323031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:38.323031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:38.323031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:38.323031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:38.324896 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:38.324896 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:38.324896 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:41:38.324896 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:41:38.324896 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:41:38.324896 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:41:38.324896 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:41:38.324896 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:41:38.324896 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 14:41:38.324896 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:38.324896 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 279/553] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:41:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:38.326946 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25f694-66fb-4011-8b1e-b2a814e0a888 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:38.328716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14043294-0fb9-4fd7-8411-23a0317a479c None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:38.568356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14043294-0fb9-4fd7-8411-23a0317a479c None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:38.644441 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7efefbc-a4e6-4c7c-af5e-2cce12f6a4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:38.645140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efefbc-a4e6-4c7c-af5e-2cce12f6a4c2 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:38.645421 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efefbc-a4e6-4c7c-af5e-2cce12f6a4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:38.645696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efefbc-a4e6-4c7c-af5e-2cce12f6a4c2 None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:38.646576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7efefbc-a4e6-4c7c-af5e-2cce12f6a4c2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:38.646884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7efefbc-a4e6-4c7c-af5e-2cce12f6a4c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:38.648347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7efefbc-a4e6-4c7c-af5e-2cce12f6a4c2 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:38.652281 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25f694-66fb-4011-8b1e-b2a814e0a888 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:38.677631 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:38.706753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 280/554] 10.222.0.78 () {56 vars in 1135 bytes} [Wed May 25 14:41:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:38.713293 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7efefbc-a4e6-4c7c-af5e-2cce12f6a4c2 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:38.726037 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e8398-d5a3-4d3e-8609-a379a9375dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:38.726037 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e8398-d5a3-4d3e-8609-a379a9375dba None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:38.726037 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e8398-d5a3-4d3e-8609-a379a9375dba None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:38.726037 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e8398-d5a3-4d3e-8609-a379a9375dba None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:38.726461 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e8398-d5a3-4d3e-8609-a379a9375dba None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:38.726461 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e8398-d5a3-4d3e-8609-a379a9375dba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:38.726461 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e8398-d5a3-4d3e-8609-a379a9375dba None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:38.734200 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:38.737617 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 275/555] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:38.745963 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e8398-d5a3-4d3e-8609-a379a9375dba None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:38.860373 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e8398-d5a3-4d3e-8609-a379a9375dba None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:38.901557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f695a-59f8-4fdb-84f9-7ad319fc0120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:38.902679 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f695a-59f8-4fdb-84f9-7ad319fc0120 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:38.903262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f695a-59f8-4fdb-84f9-7ad319fc0120 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:38.903698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f695a-59f8-4fdb-84f9-7ad319fc0120 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:38.917386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f695a-59f8-4fdb-84f9-7ad319fc0120 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:38.919751 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f695a-59f8-4fdb-84f9-7ad319fc0120 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:38.937231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-cd7f695a-59f8-4fdb-84f9-7ad319fc0120 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:41:38.937231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:38.937231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:38.937231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:38.937231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:38.937231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:38.937231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:38.937231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:38.937231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:38.937231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:38.937231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:38.937231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:38.937231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:41:38.937231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:41:38.937231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:41:38.937231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:41:38.938315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:38.938315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:38.938315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:41:38.938315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:38.938315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:41:38.938315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:38.938315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:38.938315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:38.938315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:38.938315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:38.938315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:38.938315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:38.938315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:38.938315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:38.938315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:38.938315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:38.938315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:38.938315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:38.939698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:38.939698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:38.939698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:41:38.939698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:41:38.939698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:41:38.939698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:41:38.939698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:41:38.939698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:41:38.939698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:41:38.939698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:38.939698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 276/556] 10.222.0.78 () {56 vars in 1093 bytes} [Wed May 25 14:41:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:38.944609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f695a-59f8-4fdb-84f9-7ad319fc0120 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:38.990446 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:39.021126 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7efefbc-a4e6-4c7c-af5e-2cce12f6a4c2 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:39.065897 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f695a-59f8-4fdb-84f9-7ad319fc0120 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:39.107043 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c419f-7595-4146-b787-a378d60581df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:39.107812 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c419f-7595-4146-b787-a378d60581df None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:39.108098 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c419f-7595-4146-b787-a378d60581df None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:39.108378 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c419f-7595-4146-b787-a378d60581df None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:39.109314 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c419f-7595-4146-b787-a378d60581df None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:39.109636 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c419f-7595-4146-b787-a378d60581df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:39.111469 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c419f-7595-4146-b787-a378d60581df None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:39.132717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 277/557] 10.222.0.78 () {56 vars in 1133 bytes} [Wed May 25 14:41:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:39.138289 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c419f-7595-4146-b787-a378d60581df None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:39.146747 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb1b95-cd47-4acc-b817-c377ed1e7ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:39.147417 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb1b95-cd47-4acc-b817-c377ed1e7ad6 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:39.147701 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb1b95-cd47-4acc-b817-c377ed1e7ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:39.147975 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb1b95-cd47-4acc-b817-c377ed1e7ad6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:39.246409 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:39.250717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c419f-7595-4146-b787-a378d60581df None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:39.293220 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264bf35-bb83-4f52-8a04-ef27ad0bef47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:39.294082 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264bf35-bb83-4f52-8a04-ef27ad0bef47 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:39.294375 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264bf35-bb83-4f52-8a04-ef27ad0bef47 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:39.294661 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264bf35-bb83-4f52-8a04-ef27ad0bef47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:39.307103 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264bf35-bb83-4f52-8a04-ef27ad0bef47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:39.308882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264bf35-bb83-4f52-8a04-ef27ad0bef47 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:39.335003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-1264bf35-bb83-4f52-8a04-ef27ad0bef47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:39.335003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:39.335003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:39.335003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:39.335003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:39.335003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:39.335003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:39.335003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:39.335003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:39.335003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:39.335003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:39.335003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:39.335003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:39.335003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:39.335003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:39.336029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:39.336029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:39.336029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:39.336029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:39.336029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:39.336029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:39.336029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:39.336029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:39.336029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:39.336029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:39.336029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:39.336029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:39.336029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:39.336029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:39.336029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:39.336029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:39.336029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:39.336909 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:39.336909 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:39.336909 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:39.336909 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:39.336909 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 278/558] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:41:39.343408 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264bf35-bb83-4f52-8a04-ef27ad0bef47 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:39.365997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb1b95-cd47-4acc-b817-c377ed1e7ad6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:39.367719 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb1b95-cd47-4acc-b817-c377ed1e7ad6 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:39.374973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:39.533058 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264bf35-bb83-4f52-8a04-ef27ad0bef47 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:39.564016 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 281/559] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:41:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 852 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 14:41:39.566302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744e5fe-ecc6-4446-ab46-dc8b60851588 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:39.566594 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744e5fe-ecc6-4446-ab46-dc8b60851588 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:39.566873 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744e5fe-ecc6-4446-ab46-dc8b60851588 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:39.570104 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 282/560] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:39.570998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744e5fe-ecc6-4446-ab46-dc8b60851588 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:39.582890 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d63d8d-5645-4270-9653-3b5f3d1ab01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:39.583620 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d63d8d-5645-4270-9653-3b5f3d1ab01b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:39.583903 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d63d8d-5645-4270-9653-3b5f3d1ab01b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:39.584177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d63d8d-5645-4270-9653-3b5f3d1ab01b None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:39.584922 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d63d8d-5645-4270-9653-3b5f3d1ab01b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:39.585233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d63d8d-5645-4270-9653-3b5f3d1ab01b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:39.586719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d63d8d-5645-4270-9653-3b5f3d1ab01b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:39.597465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:39.609660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 279/561] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:39.611328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d12ae-4e36-4b0e-a68b-48501d433c1c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:39.611744 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d12ae-4e36-4b0e-a68b-48501d433c1c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:39.612088 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d12ae-4e36-4b0e-a68b-48501d433c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:39.635099 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-917d12ae-4e36-4b0e-a68b-48501d433c1c None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:39.636530 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-917d12ae-4e36-4b0e-a68b-48501d433c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:39.691160 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744e5fe-ecc6-4446-ab46-dc8b60851588 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:39.694677 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:39.726665 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 280/562] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:39.726665 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d12ae-4e36-4b0e-a68b-48501d433c1c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:39.751403 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761b098-ff5c-4113-a081-be9a01a986e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:39.752201 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761b098-ff5c-4113-a081-be9a01a986e0 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:39.752483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761b098-ff5c-4113-a081-be9a01a986e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:39.753218 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761b098-ff5c-4113-a081-be9a01a986e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:39.767527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761b098-ff5c-4113-a081-be9a01a986e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:39.769019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761b098-ff5c-4113-a081-be9a01a986e0 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:39.788392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-8761b098-ff5c-4113-a081-be9a01a986e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:39.788392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:39.788392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:39.788392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:39.788392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:39.788392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:39.788392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:39.788392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:39.788392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:39.788392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:39.788392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:39.788392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:39.788392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:39.788392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:39.788392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:39.789362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:39.789362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:39.789362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:39.789362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:39.789362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:39.789362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:39.789362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:39.789362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:39.789362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:39.789362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:39.789362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:39.789362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:39.789362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:39.789362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:39.789362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:39.789362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:39.789362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:39.790595 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:39.790595 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:39.790595 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:39.790595 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:39.790595 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 283/563] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:39.790891 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e42ede-ef56-49b2-ab6d-524a80c672d2 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:39.791145 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e42ede-ef56-49b2-ab6d-524a80c672d2 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:39.791419 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e42ede-ef56-49b2-ab6d-524a80c672d2 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:39.792680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 284/564] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:39.793399 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e42ede-ef56-49b2-ab6d-524a80c672d2 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:39.842496 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d12ae-4e36-4b0e-a68b-48501d433c1c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:39.877018 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:39.883584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67585fc3-5417-4286-804a-df9a94e24b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:39.884406 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67585fc3-5417-4286-804a-df9a94e24b34 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:39.884688 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67585fc3-5417-4286-804a-df9a94e24b34 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:39.884990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67585fc3-5417-4286-804a-df9a94e24b34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:39.893673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67585fc3-5417-4286-804a-df9a94e24b34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:39.895124 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67585fc3-5417-4286-804a-df9a94e24b34 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:39.902892 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e42ede-ef56-49b2-ab6d-524a80c672d2 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:39.912402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-67585fc3-5417-4286-804a-df9a94e24b34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:39.912402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:39.912402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:39.912402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:39.912402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:39.912402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:39.912402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:39.912402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:39.912402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:39.912402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:39.912402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:39.912402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:39.912402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:41:39.912402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:41:39.912402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:41:39.913560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:41:39.913560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:39.913560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:39.913560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:39.913560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:39.913560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:39.913560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:39.913560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:39.913560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:39.913560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:39.913560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:39.913560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:39.913560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:39.913560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:39.913560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:39.913560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:39.913560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:39.914532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:41:39.914532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:41:39.914532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:41:39.914532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:41:39.914532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:41:39.914532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:41:39.914532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:39.914532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:39.914532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 281/565] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:41:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:39.919708 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd5466-9d1d-4386-8cb9-47ae273f6115 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:39.919997 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd5466-9d1d-4386-8cb9-47ae273f6115 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:39.920278 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd5466-9d1d-4386-8cb9-47ae273f6115 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:39.922281 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 282/566] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:39.923030 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd5466-9d1d-4386-8cb9-47ae273f6115 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:39.943182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ad8b3-2585-4ff2-a2e5-f052879c2417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:39.944173 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ad8b3-2585-4ff2-a2e5-f052879c2417 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:39.944291 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ad8b3-2585-4ff2-a2e5-f052879c2417 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:39.944514 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ad8b3-2585-4ff2-a2e5-f052879c2417 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:39.956082 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ad8b3-2585-4ff2-a2e5-f052879c2417 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:39.963870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-7c4ad8b3-2585-4ff2-a2e5-f052879c2417 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:39.963870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:39.963870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:39.963870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:39.963870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:39.963870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:39.963870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:39.963870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:39.963870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:39.963870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:39.963870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:39.963870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:39.963870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:39.963870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:39.963870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:39.963870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:39.965710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:39.965710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:39.965710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:39.965710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:39.965710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:39.965710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:39.965710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:39.965710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:39.965710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:39.965710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:39.965710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:39.965710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:39.965710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:39.965710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:39.965710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:39.965710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:39.967483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:39.967483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:39.967483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:39.967483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:39.967483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:39.967483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:39.967483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:39.967483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:39.967483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:39.967483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:39.967483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:39.967483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:39.967483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 285/567] 10.222.0.78 () {56 vars in 1102 bytes} [Wed May 25 14:41:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:39.967483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5612714-cbb7-4ff6-bcf2-f2f7842589b1 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:39.969338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5612714-cbb7-4ff6-bcf2-f2f7842589b1 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:39.969338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5612714-cbb7-4ff6-bcf2-f2f7842589b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:39.985795 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-d5612714-cbb7-4ff6-bcf2-f2f7842589b1 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:39.987039 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d5612714-cbb7-4ff6-bcf2-f2f7842589b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:40.000210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:40.040012 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd5466-9d1d-4386-8cb9-47ae273f6115 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:40.069361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 286/568] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:40.070272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5612714-cbb7-4ff6-bcf2-f2f7842589b1 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:40.082543 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ef356-f5f5-4c46-a827-1b72aa626590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:40.083299 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ef356-f5f5-4c46-a827-1b72aa626590 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:40.083587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ef356-f5f5-4c46-a827-1b72aa626590 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:40.083866 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ef356-f5f5-4c46-a827-1b72aa626590 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:40.084589 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ef356-f5f5-4c46-a827-1b72aa626590 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:40.084898 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ef356-f5f5-4c46-a827-1b72aa626590 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:40.086310 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ef356-f5f5-4c46-a827-1b72aa626590 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:40.099436 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 283/569] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:40.101130 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab2a91-b36f-4012-8057-ec705f91ca60 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:40.101591 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab2a91-b36f-4012-8057-ec705f91ca60 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:40.102458 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab2a91-b36f-4012-8057-ec705f91ca60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:40.129632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-32ab2a91-b36f-4012-8057-ec705f91ca60 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:40.130893 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-32ab2a91-b36f-4012-8057-ec705f91ca60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:40.181119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:40.183310 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5612714-cbb7-4ff6-bcf2-f2f7842589b1 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:40.202813 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 284/570] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:40.203688 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab2a91-b36f-4012-8057-ec705f91ca60 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:40.228034 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be010f1-6a7e-4879-b8b5-ba32447f70d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:40.228795 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be010f1-6a7e-4879-b8b5-ba32447f70d4 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:40.229078 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be010f1-6a7e-4879-b8b5-ba32447f70d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:40.229349 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be010f1-6a7e-4879-b8b5-ba32447f70d4 None admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:40.230169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be010f1-6a7e-4879-b8b5-ba32447f70d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:40.230564 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be010f1-6a7e-4879-b8b5-ba32447f70d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:40.231928 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be010f1-6a7e-4879-b8b5-ba32447f70d4 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:40.244883 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 287/571] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:41:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:40.245547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be010f1-6a7e-4879-b8b5-ba32447f70d4 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:40.258499 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:40.308934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab2a91-b36f-4012-8057-ec705f91ca60 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:40.345121 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efdb7d-9b26-483a-8449-d43014ee4887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:40.345806 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efdb7d-9b26-483a-8449-d43014ee4887 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:40.346119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efdb7d-9b26-483a-8449-d43014ee4887 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:40.346393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efdb7d-9b26-483a-8449-d43014ee4887 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:40.347158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32efdb7d-9b26-483a-8449-d43014ee4887 None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:40.347466 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32efdb7d-9b26-483a-8449-d43014ee4887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:40.349578 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32efdb7d-9b26-483a-8449-d43014ee4887 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:40.360381 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 285/572] 10.222.0.78 () {56 vars in 1103 bytes} [Wed May 25 14:41:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:40.361226 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efdb7d-9b26-483a-8449-d43014ee4887 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:40.363037 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be010f1-6a7e-4879-b8b5-ba32447f70d4 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:40.399228 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb3bf2-2145-475e-a999-89a14090149e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:40.400017 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb3bf2-2145-475e-a999-89a14090149e None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:40.400292 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb3bf2-2145-475e-a999-89a14090149e None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:40.400562 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb3bf2-2145-475e-a999-89a14090149e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:40.407659 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:40.412129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb3bf2-2145-475e-a999-89a14090149e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:40.419899 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-94fb3bf2-2145-475e-a999-89a14090149e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:40.419899 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:40.419899 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:40.419899 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:40.419899 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:40.419899 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:40.419899 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:40.419899 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:40.419899 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:40.419899 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:40.419899 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:40.419899 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:40.419899 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:40.419899 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:40.419899 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:40.419899 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:40.421253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:40.421253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:40.421253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:40.421253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:40.421253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:40.421253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:40.421253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:40.421253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:40.421253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:40.421253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:40.421253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:40.421253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:40.421253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:40.421253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:40.421253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:40.421253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:40.422222 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:40.422222 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:40.422222 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:40.422222 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:40.422222 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:40.422222 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:40.422222 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:40.422222 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:40.422222 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:40.422222 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:40.422222 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:40.422222 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:40.422222 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 288/573] 10.222.0.78 () {56 vars in 1102 bytes} [Wed May 25 14:41:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:40.424141 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb3bf2-2145-475e-a999-89a14090149e None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:40.469028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efdb7d-9b26-483a-8449-d43014ee4887 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:40.493123 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:40.503996 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07beeb-e298-4867-838f-f13fb15924c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:40.505212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07beeb-e298-4867-838f-f13fb15924c9 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:40.505558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07beeb-e298-4867-838f-f13fb15924c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:40.505760 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07beeb-e298-4867-838f-f13fb15924c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:40.513305 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07beeb-e298-4867-838f-f13fb15924c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:40.515291 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07beeb-e298-4867-838f-f13fb15924c9 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:40.530039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-1a07beeb-e298-4867-838f-f13fb15924c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:40.530039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:40.530039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:40.530039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:40.530039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:40.530039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:40.530039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:40.530039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:40.530039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:40.530039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:40.530039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:40.530039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:40.530039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:41:40.530039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:41:40.530039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:41:40.531858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:41:40.531858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:40.531858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:40.531858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:40.531858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:40.531858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:40.531858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:40.531858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:40.531858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:40.531858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:40.531858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:40.531858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:40.531858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:40.531858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:40.531858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:40.531858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:40.531858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:40.533112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:41:40.533112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:41:40.533112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:41:40.533112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:41:40.533112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:41:40.533112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:41:40.533112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:40.533112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:40.533540 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb3bf2-2145-475e-a999-89a14090149e None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:40.533616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 286/574] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:41:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:40.533616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07beeb-e298-4867-838f-f13fb15924c9 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:40.565839 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed02149-0672-4388-8d6b-8a53f1c883aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:40.566663 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed02149-0672-4388-8d6b-8a53f1c883aa None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:40.566943 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed02149-0672-4388-8d6b-8a53f1c883aa None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:40.567215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed02149-0672-4388-8d6b-8a53f1c883aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:40.578238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed02149-0672-4388-8d6b-8a53f1c883aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:40.585924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-5ed02149-0672-4388-8d6b-8a53f1c883aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:40.585924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:40.585924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:40.585924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:40.585924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:40.585924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:40.585924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:40.585924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:40.585924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:40.585924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:40.585924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:40.585924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:40.585924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:40.585924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:40.585924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:40.585924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:40.587113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:40.587113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:40.587113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:40.587113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:40.587113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:40.587113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:40.587113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:40.587113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:40.587113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:40.587113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:40.587113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:40.587113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:40.587113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:40.587113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:40.587113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:40.587113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:40.588169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:40.588169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:40.588169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:40.588169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:40.588169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:40.588169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:40.588169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:40.588169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:40.588169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:40.588169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:40.588169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:40.588169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:40.588169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 289/575] 10.222.0.78 () {56 vars in 1102 bytes} [Wed May 25 14:41:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:40.588169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed02149-0672-4388-8d6b-8a53f1c883aa None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:40.642969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07beeb-e298-4867-838f-f13fb15924c9 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:40.644581 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:40.679445 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb781e4f-1fcc-4ab0-a88d-927884fb415d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:40.680333 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb781e4f-1fcc-4ab0-a88d-927884fb415d None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:40.680622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb781e4f-1fcc-4ab0-a88d-927884fb415d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:40.680902 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb781e4f-1fcc-4ab0-a88d-927884fb415d None admin] PATH_INFO: `/v3/projects/908a13d2b49544fbaf760b07e213bb7d/users/e34e1d7ffbac49b0bde45829af0c6925/roles/336f27e7f8624a30b30cef9acada97a0` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:40.695958 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed02149-0672-4388-8d6b-8a53f1c883aa None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:40.696179 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb781e4f-1fcc-4ab0-a88d-927884fb415d None admin] RBAC: Authorizing `identity:create_grant(project_id=908a13d2b49544fbaf760b07e213bb7d, user_id=e34e1d7ffbac49b0bde45829af0c6925, role_id=336f27e7f8624a30b30cef9acada97a0)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:40.697657 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb781e4f-1fcc-4ab0-a88d-927884fb415d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:40.712180 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 287/576] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:41:40 2022] PUT /identity/v3/projects/908a13d2b49544fbaf760b07e213bb7d/users/e34e1d7ffbac49b0bde45829af0c6925/roles/336f27e7f8624a30b30cef9acada97a0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:40.713018 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb781e4f-1fcc-4ab0-a88d-927884fb415d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:40.725145 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:40.736497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581c5cc-e59d-449b-a9cb-b62176a16644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:40.737964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581c5cc-e59d-449b-a9cb-b62176a16644 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:40.738290 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581c5cc-e59d-449b-a9cb-b62176a16644 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:40.738572 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581c5cc-e59d-449b-a9cb-b62176a16644 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:40.751751 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b581c5cc-e59d-449b-a9cb-b62176a16644 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:40.753307 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b581c5cc-e59d-449b-a9cb-b62176a16644 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:40.772942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-b581c5cc-e59d-449b-a9cb-b62176a16644 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:40.772942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:40.772942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:40.772942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:40.772942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:40.772942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:40.772942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:40.772942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:40.772942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:40.772942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:40.772942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:40.772942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:40.772942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:40.772942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:40.772942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:40.773912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:40.773912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:40.773912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:40.773912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:40.773912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:40.773912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:40.773912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:40.773912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:40.773912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:40.773912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:40.773912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:40.773912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:40.773912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:40.773912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:40.773912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:40.773912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:40.773912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:40.774857 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:40.774857 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:40.774857 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:40.774857 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:40.774857 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 290/577] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:40.775640 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581c5cc-e59d-449b-a9cb-b62176a16644 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:40.843352 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb781e4f-1fcc-4ab0-a88d-927884fb415d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:40.887208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bf479-0faf-4897-bf41-3de313dd6f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:40.888745 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bf479-0faf-4897-bf41-3de313dd6f53 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:40.889234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bf479-0faf-4897-bf41-3de313dd6f53 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:40.890498 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bf479-0faf-4897-bf41-3de313dd6f53 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:40.891707 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2bf479-0faf-4897-bf41-3de313dd6f53 None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:40.892536 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2bf479-0faf-4897-bf41-3de313dd6f53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:40.894743 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2bf479-0faf-4897-bf41-3de313dd6f53 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:40.898404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581c5cc-e59d-449b-a9cb-b62176a16644 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:40.914116 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 288/578] 10.222.0.78 () {56 vars in 1103 bytes} [Wed May 25 14:41:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:40.915283 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bf479-0faf-4897-bf41-3de313dd6f53 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:40.940565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:40.942048 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05e034-8eb7-41ab-a301-4d0a16973fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:40.942390 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05e034-8eb7-41ab-a301-4d0a16973fe2 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:40.942719 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05e034-8eb7-41ab-a301-4d0a16973fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:40.943025 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05e034-8eb7-41ab-a301-4d0a16973fe2 None admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:40.943941 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec05e034-8eb7-41ab-a301-4d0a16973fe2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:40.944283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec05e034-8eb7-41ab-a301-4d0a16973fe2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:40.945740 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec05e034-8eb7-41ab-a301-4d0a16973fe2 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:40.959847 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 291/579] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:41:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:40.960695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05e034-8eb7-41ab-a301-4d0a16973fe2 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:41.006184 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:41.045210 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bf479-0faf-4897-bf41-3de313dd6f53 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:41.089660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee9b24-9758-425c-ad7e-786ce07fa2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:41.089915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05e034-8eb7-41ab-a301-4d0a16973fe2 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:41.090387 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee9b24-9758-425c-ad7e-786ce07fa2b7 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:41.090721 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee9b24-9758-425c-ad7e-786ce07fa2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:41.091021 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee9b24-9758-425c-ad7e-786ce07fa2b7 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:41.091789 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee9b24-9758-425c-ad7e-786ce07fa2b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:41.092104 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee9b24-9758-425c-ad7e-786ce07fa2b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:41.094310 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee9b24-9758-425c-ad7e-786ce07fa2b7 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:41.108481 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 289/580] 10.222.0.78 () {56 vars in 1103 bytes} [Wed May 25 14:41:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:41.108481 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee9b24-9758-425c-ad7e-786ce07fa2b7 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:41.138863 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19456fb5-4f2b-4b77-93ed-85cef5107fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:41.140460 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19456fb5-4f2b-4b77-93ed-85cef5107fda None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:41.140848 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19456fb5-4f2b-4b77-93ed-85cef5107fda None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:41.141240 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19456fb5-4f2b-4b77-93ed-85cef5107fda None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:41.142406 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19456fb5-4f2b-4b77-93ed-85cef5107fda None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:41.142855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19456fb5-4f2b-4b77-93ed-85cef5107fda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:41.145058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19456fb5-4f2b-4b77-93ed-85cef5107fda None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:41.170068 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 292/581] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:41.170068 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19456fb5-4f2b-4b77-93ed-85cef5107fda None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:41.210221 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:41.244824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee9b24-9758-425c-ad7e-786ce07fa2b7 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:41.280732 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d282a0-278d-4d93-b184-13edb60652c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:41.281446 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d282a0-278d-4d93-b184-13edb60652c4 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:41.281728 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d282a0-278d-4d93-b184-13edb60652c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:41.282002 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d282a0-278d-4d93-b184-13edb60652c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:41.294223 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d282a0-278d-4d93-b184-13edb60652c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:41.296337 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d282a0-278d-4d93-b184-13edb60652c4 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:41.309036 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:41.314239 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19456fb5-4f2b-4b77-93ed-85cef5107fda None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:41.315119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-d3d282a0-278d-4d93-b184-13edb60652c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:41.315119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:41.315119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:41.315119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:41.315119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:41.315119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:41.315119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:41.315119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:41.315119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:41.315119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:41.315119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:41.315119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:41.315119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:41.315119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:41.315119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:41.316464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:41.316464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:41.316464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:41.316464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:41.316464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:41.316464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:41.316464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:41.316464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:41.316464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:41.316464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:41.316464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:41.316464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:41.316464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:41.316464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:41.316464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:41.316464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:41.316464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:41.317324 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:41.317324 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:41.317324 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:41.317324 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:41.317324 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 290/582] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:41.317602 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d282a0-278d-4d93-b184-13edb60652c4 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:41.359228 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d1631-bc8c-416d-8c64-5b0a8e5a3864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:41.359883 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d1631-bc8c-416d-8c64-5b0a8e5a3864 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:41.360162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d1631-bc8c-416d-8c64-5b0a8e5a3864 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:41.360433 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d1631-bc8c-416d-8c64-5b0a8e5a3864 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:41.361697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d1631-bc8c-416d-8c64-5b0a8e5a3864 None admin] RBAC: Adding query filter params (role.id=336f27e7f8624a30b30cef9acada97a0, scope.project.id=908a13d2b49544fbaf760b07e213bb7d, user.id=fbe4a2d106d84fc9ac65c046ba675353) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:41.362032 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d1631-bc8c-416d-8c64-5b0a8e5a3864 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:41.363834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d1631-bc8c-416d-8c64-5b0a8e5a3864 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:41.370566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 293/583] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:41:41 2022] GET /identity/v3/role_assignments?user.id=fbe4a2d106d84fc9ac65c046ba675353&scope.project.id=908a13d2b49544fbaf760b07e213bb7d&role.id=336f27e7f8624a30b30cef9acada97a0 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:41.370566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d1631-bc8c-416d-8c64-5b0a8e5a3864 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:41.447766 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d282a0-278d-4d93-b184-13edb60652c4 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:41.479809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:41.490701 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fa7da-d86e-4c7c-8b1f-f4944c818c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:41.491619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fa7da-d86e-4c7c-8b1f-f4944c818c7a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:41.491975 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fa7da-d86e-4c7c-8b1f-f4944c818c7a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:41.492315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fa7da-d86e-4c7c-8b1f-f4944c818c7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:41.499752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d1631-bc8c-416d-8c64-5b0a8e5a3864 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:41.506173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758fa7da-d86e-4c7c-8b1f-f4944c818c7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:41.507806 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758fa7da-d86e-4c7c-8b1f-f4944c818c7a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:41.527862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-758fa7da-d86e-4c7c-8b1f-f4944c818c7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:41.527862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:41.527862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:41.527862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:41.527862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:41.527862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:41.527862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:41.527862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:41.527862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:41.527862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:41.527862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:41.527862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:41.527862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:41.527862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:41.527862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:41.528809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:41.528809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:41.528809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:41.528809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:41.528809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:41.528809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:41.528809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:41.528809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:41.528809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:41.528809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:41.528809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:41.528809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:41.528809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:41.528809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:41.528809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:41.528809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:41.528809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:41.529680 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:41.529680 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:41.529680 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:41.529680 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:41.534320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 291/584] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:41.535256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fa7da-d86e-4c7c-8b1f-f4944c818c7a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:41.540799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccaa28e-b7c2-46b8-968c-797efb63d514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:41.541623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccaa28e-b7c2-46b8-968c-797efb63d514 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:41.541901 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccaa28e-b7c2-46b8-968c-797efb63d514 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:41.542196 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccaa28e-b7c2-46b8-968c-797efb63d514 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:41.551839 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccaa28e-b7c2-46b8-968c-797efb63d514 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:41.553320 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccaa28e-b7c2-46b8-968c-797efb63d514 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:41.569476 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-3ccaa28e-b7c2-46b8-968c-797efb63d514 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 25 14:41:41.569476 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:41.569476 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:41.569476 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:41.569476 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:41.569476 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:41.569476 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:41.569476 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:41.569476 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:41.569476 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:41.569476 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:41.569476 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:41.569476 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:41:41.569476 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:41:41.569476 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:41:41.569476 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:41:41.571698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:41.571698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:41.571698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:41:41.571698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:41.571698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:41:41.571698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:41.571698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:41.571698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:41.571698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:41.571698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:41.571698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:41.571698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:41.571698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:41.571698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:41.571698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:41.571698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:41.571698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:41.571698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:41.573872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:41.573872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:41.573872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:41:41.573872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:41:41.573872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:41:41.573872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:41:41.573872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:41:41.573872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:41:41.573872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 25 14:41:41.573872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:41.573872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 294/585] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:41:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:41.573872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccaa28e-b7c2-46b8-968c-797efb63d514 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:41.605160 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:41.661179 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fa7da-d86e-4c7c-8b1f-f4944c818c7a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:41.697379 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ee2d0-2d64-4fb1-94ce-c3b76437b12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:41.697932 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccaa28e-b7c2-46b8-968c-797efb63d514 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:41.698178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ee2d0-2d64-4fb1-94ce-c3b76437b12c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:41.698423 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ee2d0-2d64-4fb1-94ce-c3b76437b12c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:41.698699 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ee2d0-2d64-4fb1-94ce-c3b76437b12c None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:41.699484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ee2d0-2d64-4fb1-94ce-c3b76437b12c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:41.699788 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ee2d0-2d64-4fb1-94ce-c3b76437b12c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:41.701232 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ee2d0-2d64-4fb1-94ce-c3b76437b12c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:41.719753 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 292/586] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:41.720776 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ee2d0-2d64-4fb1-94ce-c3b76437b12c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:41.740534 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e94a94-b606-49c5-b160-03bb6d0203a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:41.741585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e94a94-b606-49c5-b160-03bb6d0203a1 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:41.741914 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e94a94-b606-49c5-b160-03bb6d0203a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:41.742257 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e94a94-b606-49c5-b160-03bb6d0203a1 None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:41.743016 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e94a94-b606-49c5-b160-03bb6d0203a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:41.743320 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e94a94-b606-49c5-b160-03bb6d0203a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:41.745348 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e94a94-b606-49c5-b160-03bb6d0203a1 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:41.763545 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 295/587] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:41.765046 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e94a94-b606-49c5-b160-03bb6d0203a1 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:41.771463 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:41.858130 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ee2d0-2d64-4fb1-94ce-c3b76437b12c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:41.870387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:41.891246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e94a94-b606-49c5-b160-03bb6d0203a1 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:41.910905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a922d9-723c-4660-8016-e49ef85ee160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:41.912539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a922d9-723c-4660-8016-e49ef85ee160 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:41.913261 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a922d9-723c-4660-8016-e49ef85ee160 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:41.913727 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a922d9-723c-4660-8016-e49ef85ee160 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:41.914987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a922d9-723c-4660-8016-e49ef85ee160 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:41.915781 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a922d9-723c-4660-8016-e49ef85ee160 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:41.917429 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a922d9-723c-4660-8016-e49ef85ee160 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:41.942174 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ea681-8802-45ac-823d-93094a7166da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:41.942174 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ea681-8802-45ac-823d-93094a7166da None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:41.942174 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ea681-8802-45ac-823d-93094a7166da None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:41.942546 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ea681-8802-45ac-823d-93094a7166da None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:41.948889 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 293/588] 10.222.0.78 () {56 vars in 1137 bytes} [Wed May 25 14:41:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:41.950076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a922d9-723c-4660-8016-e49ef85ee160 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:41.953203 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ea681-8802-45ac-823d-93094a7166da None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:41.954846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ea681-8802-45ac-823d-93094a7166da None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:41.970659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-7a3ea681-8802-45ac-823d-93094a7166da None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 25 14:41:41.970659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:41.970659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:41.970659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:41.970659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:41.970659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:41.970659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:41.970659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:41.970659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:41.970659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:41.970659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:41.970659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:41.970659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:41:41.970659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:41:41.970659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:41:41.970659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:41:41.971866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:41.971866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:41.971866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:41:41.971866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:41.971866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:41:41.971866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:41.971866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:41.971866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:41.971866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:41.971866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:41.971866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:41.971866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:41.971866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:41.971866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:41.971866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:41.971866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:41.971866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:41.971866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:41.973097 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:41.973097 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:41.973097 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:41:41.973097 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:41:41.973097 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:41:41.973097 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:41:41.973097 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:41:41.973097 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:41:41.973097 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 25 14:41:41.973097 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:41.978151 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 296/589] 10.222.0.78 () {56 vars in 1102 bytes} [Wed May 25 14:41:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:41.979139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ea681-8802-45ac-823d-93094a7166da None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:42.100469 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:42.104759 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a922d9-723c-4660-8016-e49ef85ee160 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:42.118372 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ea681-8802-45ac-823d-93094a7166da None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:42.145272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffba8b8-168b-4dff-8d09-7bb21b98b6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:42.146208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffba8b8-168b-4dff-8d09-7bb21b98b6ec None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:42.146604 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffba8b8-168b-4dff-8d09-7bb21b98b6ec None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:42.146979 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffba8b8-168b-4dff-8d09-7bb21b98b6ec None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:42.157810 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffba8b8-168b-4dff-8d09-7bb21b98b6ec None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:42.159512 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffba8b8-168b-4dff-8d09-7bb21b98b6ec None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:42.166714 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a98c8f-4dbd-434d-9552-714757cf2caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:42.168215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a98c8f-4dbd-434d-9552-714757cf2caf None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:42.168723 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a98c8f-4dbd-434d-9552-714757cf2caf None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:42.169245 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a98c8f-4dbd-434d-9552-714757cf2caf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:42.176612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-fffba8b8-168b-4dff-8d09-7bb21b98b6ec None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 14:41:42.176612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:42.176612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:42.176612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:42.176612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:42.176612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:42.176612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:42.176612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:42.176612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:42.176612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:42.176612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:42.176612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:42.176612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:41:42.176612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:41:42.176612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:41:42.176612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:41:42.177622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:42.177622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:42.177622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:41:42.177622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:42.177622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:41:42.177622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:42.177622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:42.177622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:42.177622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:42.177622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:42.177622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:42.177622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:42.177622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:42.177622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:42.177622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:42.177622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:42.177622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:42.177622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:42.178611 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:42.178611 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:42.178611 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:41:42.178611 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:41:42.178611 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:41:42.178611 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:41:42.178611 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:41:42.178611 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:41:42.178611 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 14:41:42.178611 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:42.182212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 294/590] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:41:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:42.183555 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffba8b8-168b-4dff-8d09-7bb21b98b6ec None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:42.192284 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a98c8f-4dbd-434d-9552-714757cf2caf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:42.195624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a98c8f-4dbd-434d-9552-714757cf2caf None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:42.198122 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:42.217387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-e9a98c8f-4dbd-434d-9552-714757cf2caf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:42.217387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:42.217387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:42.217387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:42.217387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:42.217387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:42.217387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:42.217387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:42.217387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:42.217387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:42.217387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:42.217387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:42.217387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:42.217387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:42.217387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:42.219050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:42.219050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:42.219050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:42.219050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:42.219050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:42.219050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:42.219050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:42.219050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:42.219050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:42.219050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:42.219050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:42.219050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:42.219050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:42.219050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:42.219050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:42.219050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:42.219050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:42.220103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:42.220103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:42.220103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:42.220103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:42.226418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 297/591] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:42.227634 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a98c8f-4dbd-434d-9552-714757cf2caf None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:42.339607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffba8b8-168b-4dff-8d09-7bb21b98b6ec None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:42.380472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed16a1e-0c41-4be3-ae25-042d84b162b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:42.381240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed16a1e-0c41-4be3-ae25-042d84b162b1 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:42.381597 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed16a1e-0c41-4be3-ae25-042d84b162b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:42.381940 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed16a1e-0c41-4be3-ae25-042d84b162b1 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:42.382914 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed16a1e-0c41-4be3-ae25-042d84b162b1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:42.383291 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed16a1e-0c41-4be3-ae25-042d84b162b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:42.384947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed16a1e-0c41-4be3-ae25-042d84b162b1 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:42.411591 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 295/592] 10.222.0.78 () {56 vars in 1139 bytes} [Wed May 25 14:41:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:42.411591 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed16a1e-0c41-4be3-ae25-042d84b162b1 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:42.417385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a98c8f-4dbd-434d-9552-714757cf2caf None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:42.428084 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:42.482500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cad8f1-5867-45ed-b983-59ed901598ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:42.484479 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cad8f1-5867-45ed-b983-59ed901598ae None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:42.485069 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cad8f1-5867-45ed-b983-59ed901598ae None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:42.485596 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cad8f1-5867-45ed-b983-59ed901598ae None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:42.487520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cad8f1-5867-45ed-b983-59ed901598ae None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:42.488429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cad8f1-5867-45ed-b983-59ed901598ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:42.491480 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cad8f1-5867-45ed-b983-59ed901598ae None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:42.520600 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 298/593] 10.222.0.78 () {56 vars in 1137 bytes} [Wed May 25 14:41:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:42.522325 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cad8f1-5867-45ed-b983-59ed901598ae None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:42.522698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed16a1e-0c41-4be3-ae25-042d84b162b1 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:42.564656 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20725bf2-6db0-4b71-8b25-f8556742e8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:42.564656 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20725bf2-6db0-4b71-8b25-f8556742e8af None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:42.564934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20725bf2-6db0-4b71-8b25-f8556742e8af None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:42.565173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20725bf2-6db0-4b71-8b25-f8556742e8af None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:42.565803 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20725bf2-6db0-4b71-8b25-f8556742e8af None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:42.566170 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20725bf2-6db0-4b71-8b25-f8556742e8af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:42.567604 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20725bf2-6db0-4b71-8b25-f8556742e8af None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:42.585465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 296/594] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:42.585465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20725bf2-6db0-4b71-8b25-f8556742e8af None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:42.592009 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:42.665101 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cad8f1-5867-45ed-b983-59ed901598ae None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:42.688119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:42.710733 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b755cc-335b-478d-8d98-e1f9f76024d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:42.711890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b755cc-335b-478d-8d98-e1f9f76024d5 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:42.712379 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b755cc-335b-478d-8d98-e1f9f76024d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:42.712880 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b755cc-335b-478d-8d98-e1f9f76024d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:42.723388 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20725bf2-6db0-4b71-8b25-f8556742e8af None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:42.725003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b755cc-335b-478d-8d98-e1f9f76024d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:42.726822 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b755cc-335b-478d-8d98-e1f9f76024d5 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:42.746622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-45b755cc-335b-478d-8d98-e1f9f76024d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:42.746622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:42.746622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:42.746622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:42.746622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:42.746622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:42.746622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:42.746622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:42.746622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:42.746622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:42.746622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:42.746622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:42.746622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:42.746622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:42.746622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:42.747598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:42.747598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:42.747598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:42.747598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:42.747598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:42.747598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:42.747598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:42.747598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:42.747598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:42.747598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:42.747598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:42.747598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:42.747598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:42.747598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:42.747598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:42.747598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:42.747598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:42.748484 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:42.748484 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:42.748484 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:42.748484 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:42.755912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 299/595] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:42.757215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b755cc-335b-478d-8d98-e1f9f76024d5 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:42.767578 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc774d9-3db7-4605-8686-fa3bf54786bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:42.768386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc774d9-3db7-4605-8686-fa3bf54786bb None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:42.768671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc774d9-3db7-4605-8686-fa3bf54786bb None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:42.768950 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc774d9-3db7-4605-8686-fa3bf54786bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:42.780991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc774d9-3db7-4605-8686-fa3bf54786bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:42.782570 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc774d9-3db7-4605-8686-fa3bf54786bb None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:42.802386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-6bc774d9-3db7-4605-8686-fa3bf54786bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:42.802386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:42.802386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:42.802386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:42.802386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:42.802386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:42.802386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:42.802386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:42.802386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:42.802386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:42.802386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:42.802386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:42.802386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:42.802386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:42.802386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:42.803360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:42.803360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:42.803360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:42.803360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:42.803360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:42.803360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:42.803360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:42.803360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:42.803360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:42.803360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:42.803360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:42.803360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:42.803360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:42.803360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:42.803360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:42.803360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:42.803360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:42.804250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:42.804250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:42.804250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:42.804250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:42.804250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 297/596] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:42.809325 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc774d9-3db7-4605-8686-fa3bf54786bb None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:42.879237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b755cc-335b-478d-8d98-e1f9f76024d5 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:42.882694 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:42.924638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc774d9-3db7-4605-8686-fa3bf54786bb None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:42.925323 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f7ddc-78f6-48c8-aa2a-7e3ff0686851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:42.926429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f7ddc-78f6-48c8-aa2a-7e3ff0686851 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:42.926932 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f7ddc-78f6-48c8-aa2a-7e3ff0686851 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:42.927407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f7ddc-78f6-48c8-aa2a-7e3ff0686851 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:42.935451 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f7ddc-78f6-48c8-aa2a-7e3ff0686851 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:42.937249 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f7ddc-78f6-48c8-aa2a-7e3ff0686851 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:42.951973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-f05f7ddc-78f6-48c8-aa2a-7e3ff0686851 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:42.951973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:42.951973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:42.951973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:42.951973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:42.951973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:42.951973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:42.951973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:42.951973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:42.951973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:42.951973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:42.951973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:42.951973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:41:42.951973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:41:42.951973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:41:42.952911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:41:42.952911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:42.952911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:42.952911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:42.952911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:42.952911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:42.952911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:42.952911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:42.952911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:42.952911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:42.952911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:42.952911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:42.952911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:42.952911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:42.952911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:42.952911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:42.952911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:42.953850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:41:42.953850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:41:42.953850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:41:42.953850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:41:42.953850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:41:42.953850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:41:42.953850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:42.953850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:42.955331 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 300/597] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:41:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:42.956340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f7ddc-78f6-48c8-aa2a-7e3ff0686851 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:42.960322 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:42.969818 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1244d8-eb92-4665-8bd7-0ab63bbe6160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:42.970527 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1244d8-eb92-4665-8bd7-0ab63bbe6160 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:42.970807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1244d8-eb92-4665-8bd7-0ab63bbe6160 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:42.971081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1244d8-eb92-4665-8bd7-0ab63bbe6160 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:42.971800 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1244d8-eb92-4665-8bd7-0ab63bbe6160 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:42.972104 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1244d8-eb92-4665-8bd7-0ab63bbe6160 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:42.973653 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1244d8-eb92-4665-8bd7-0ab63bbe6160 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:42.993278 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 298/598] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:42.993278 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1244d8-eb92-4665-8bd7-0ab63bbe6160 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:43.091436 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f7ddc-78f6-48c8-aa2a-7e3ff0686851 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:43.095486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1244d8-eb92-4665-8bd7-0ab63bbe6160 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:43.132548 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d196c-086d-4656-a1cd-604aeb7a2ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:43.132762 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a0644-b7d6-413d-9e72-ef6f51b1801a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:43.133216 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a0644-b7d6-413d-9e72-ef6f51b1801a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:43.133319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d196c-086d-4656-a1cd-604aeb7a2ab1 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:43.133494 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d196c-086d-4656-a1cd-604aeb7a2ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:43.133590 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a0644-b7d6-413d-9e72-ef6f51b1801a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:43.133774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a0644-b7d6-413d-9e72-ef6f51b1801a None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:43.133879 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d196c-086d-4656-a1cd-604aeb7a2ab1 None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:43.134541 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d196c-086d-4656-a1cd-604aeb7a2ab1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:43.134780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a0644-b7d6-413d-9e72-ef6f51b1801a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:43.134853 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d196c-086d-4656-a1cd-604aeb7a2ab1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:43.134922 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a0644-b7d6-413d-9e72-ef6f51b1801a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:43.136216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d196c-086d-4656-a1cd-604aeb7a2ab1 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:43.136312 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a0644-b7d6-413d-9e72-ef6f51b1801a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:43.153717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 301/599] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:43.153717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d196c-086d-4656-a1cd-604aeb7a2ab1 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:43.153977 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 299/600] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:41:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:43.153977 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67b15d-1888-4dd4-bcdc-c938a8f5f4ad None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:43.153977 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67b15d-1888-4dd4-bcdc-c938a8f5f4ad None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:43.153977 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67b15d-1888-4dd4-bcdc-c938a8f5f4ad None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:43.154887 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 300/601] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:43.156060 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67b15d-1888-4dd4-bcdc-c938a8f5f4ad None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:43.190053 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:43.220106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:43.271805 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d196c-086d-4656-a1cd-604aeb7a2ab1 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:43.281318 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67b15d-1888-4dd4-bcdc-c938a8f5f4ad None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:43.307020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb8af0-4e81-438a-b5c9-2f0ed43b0881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:43.307818 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb8af0-4e81-438a-b5c9-2f0ed43b0881 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:43.308096 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb8af0-4e81-438a-b5c9-2f0ed43b0881 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:43.308361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb8af0-4e81-438a-b5c9-2f0ed43b0881 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:43.315523 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb8af0-4e81-438a-b5c9-2f0ed43b0881 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:43.317011 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb8af0-4e81-438a-b5c9-2f0ed43b0881 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:43.324770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac88719-a38f-497d-85ae-16b4e9541672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:43.325918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac88719-a38f-497d-85ae-16b4e9541672 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:43.326398 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac88719-a38f-497d-85ae-16b4e9541672 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:43.326819 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac88719-a38f-497d-85ae-16b4e9541672 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:43.335691 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac88719-a38f-497d-85ae-16b4e9541672 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:43.337432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac88719-a38f-497d-85ae-16b4e9541672 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:43.337809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-edfb8af0-4e81-438a-b5c9-2f0ed43b0881 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:43.337809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:43.337809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:43.337809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:43.337809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:43.337809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:43.337809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:43.337809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:43.337809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:43.337809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:43.337809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:43.337809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:43.337809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:41:43.337809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:41:43.337809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:41:43.338762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:41:43.338762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:43.338762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:43.338762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:43.338762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:43.338762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:43.338762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:43.338762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:43.338762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:43.338762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:43.338762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:43.338762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:43.338762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:43.338762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:43.338762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:43.338762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:43.338762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:43.346067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:41:43.346067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:41:43.346067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:41:43.346067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:41:43.346067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:41:43.346067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:41:43.346067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:43.346067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:43.346067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 302/602] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:41:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:43.346067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb8af0-4e81-438a-b5c9-2f0ed43b0881 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:43.356537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-fac88719-a38f-497d-85ae-16b4e9541672 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:43.356537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:43.356537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:43.356537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:43.356537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:43.356537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:43.356537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:43.356537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:43.356537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:43.356537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:43.356537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:43.356537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:43.356537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:41:43.356537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:41:43.356537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:41:43.357882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:41:43.357882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:43.357882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:43.357882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:43.357882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:43.357882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:43.357882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:43.357882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:43.357882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:43.357882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:43.357882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:43.357882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:43.357882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:43.357882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:43.357882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:43.357882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:43.357882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:43.359008 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:41:43.359008 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:41:43.359008 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:41:43.359008 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:41:43.359008 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:41:43.359008 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:41:43.359008 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:43.359008 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:43.359008 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 301/603] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:41:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:43.366662 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49c237-af23-43d2-a15e-80aea1c9d767 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:43.366662 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49c237-af23-43d2-a15e-80aea1c9d767 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:43.366662 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49c237-af23-43d2-a15e-80aea1c9d767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:43.381201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-ac49c237-af23-43d2-a15e-80aea1c9d767 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:43.382443 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ac49c237-af23-43d2-a15e-80aea1c9d767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:43.442514 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:43.457990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 302/604] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:43.458895 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49c237-af23-43d2-a15e-80aea1c9d767 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:43.464666 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb8af0-4e81-438a-b5c9-2f0ed43b0881 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:43.504934 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97795259-abda-48c5-87a8-391662095d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:43.506253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97795259-abda-48c5-87a8-391662095d74 None admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:43.506716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97795259-abda-48c5-87a8-391662095d74 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:43.507166 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97795259-abda-48c5-87a8-391662095d74 None admin] PATH_INFO: `/v3/projects/908a13d2b49544fbaf760b07e213bb7d/users/88cbd76f70d447c4ba92777b2c09821d/roles/336f27e7f8624a30b30cef9acada97a0` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:43.511157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:43.528006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97795259-abda-48c5-87a8-391662095d74 None admin] RBAC: Authorizing `identity:create_grant(project_id=908a13d2b49544fbaf760b07e213bb7d, user_id=88cbd76f70d447c4ba92777b2c09821d, role_id=336f27e7f8624a30b30cef9acada97a0)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:43.529568 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97795259-abda-48c5-87a8-391662095d74 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:43.548109 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 303/605] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:41:43 2022] PUT /identity/v3/projects/908a13d2b49544fbaf760b07e213bb7d/users/88cbd76f70d447c4ba92777b2c09821d/roles/336f27e7f8624a30b30cef9acada97a0 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:43.548977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97795259-abda-48c5-87a8-391662095d74 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:43.612858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49c237-af23-43d2-a15e-80aea1c9d767 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:43.657166 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a580d-329b-4773-82e0-299b56cc4f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:43.658344 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a580d-329b-4773-82e0-299b56cc4f84 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:43.658855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a580d-329b-4773-82e0-299b56cc4f84 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:43.659350 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a580d-329b-4773-82e0-299b56cc4f84 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:43.660512 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a580d-329b-4773-82e0-299b56cc4f84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:43.661192 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a580d-329b-4773-82e0-299b56cc4f84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:43.663399 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a580d-329b-4773-82e0-299b56cc4f84 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:43.682550 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 303/606] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:41:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:43.684081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a580d-329b-4773-82e0-299b56cc4f84 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:43.704541 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97795259-abda-48c5-87a8-391662095d74 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:43.749698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b0d02-8ec9-46a8-84d3-39af376651ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:43.750366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b0d02-8ec9-46a8-84d3-39af376651ba None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:43.750649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b0d02-8ec9-46a8-84d3-39af376651ba None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:43.750893 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b0d02-8ec9-46a8-84d3-39af376651ba None admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:43.751675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b0d02-8ec9-46a8-84d3-39af376651ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:43.751981 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b0d02-8ec9-46a8-84d3-39af376651ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:43.753422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b0d02-8ec9-46a8-84d3-39af376651ba None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:43.755442 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:43.769792 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 304/607] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:41:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:43.770678 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b0d02-8ec9-46a8-84d3-39af376651ba None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:43.822884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:43.828693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a580d-329b-4773-82e0-299b56cc4f84 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:43.866652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b19a3a4-1d9c-4667-acc6-9dfd643217e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:43.867621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19a3a4-1d9c-4667-acc6-9dfd643217e5 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:43.867851 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19a3a4-1d9c-4667-acc6-9dfd643217e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:43.868127 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19a3a4-1d9c-4667-acc6-9dfd643217e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:43.881138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b19a3a4-1d9c-4667-acc6-9dfd643217e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:43.884201 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b0d02-8ec9-46a8-84d3-39af376651ba None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:43.889353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-2b19a3a4-1d9c-4667-acc6-9dfd643217e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:43.889353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:43.889353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:43.889353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:43.889353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:43.889353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:43.889353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:43.889353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:43.889353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:43.889353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:43.889353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:43.889353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:43.889353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:43.889353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:43.889353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:43.889353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:43.891347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:43.891347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:43.891347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:43.891347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:43.891347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:43.891347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:43.891347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:43.891347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:43.891347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:43.891347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:43.891347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:43.891347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:43.891347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:43.891347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:43.891347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:43.891347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:43.892262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:43.892262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:43.892262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:43.892262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:43.892262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:43.892262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:43.892262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:43.892262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:43.892262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:43.892262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:43.892262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:43.892262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:43.892262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 304/608] 10.222.0.78 () {56 vars in 1102 bytes} [Wed May 25 14:41:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:43.893155 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b7f61-48ce-44ff-90d8-a45c59113e97 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:43.893451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b7f61-48ce-44ff-90d8-a45c59113e97 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:43.893640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b7f61-48ce-44ff-90d8-a45c59113e97 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:43.899371 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 305/609] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:43.900250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b7f61-48ce-44ff-90d8-a45c59113e97 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:43.929435 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae82630-b69c-4768-bad0-7d8925d0f835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:43.930370 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae82630-b69c-4768-bad0-7d8925d0f835 None admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:43.930711 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae82630-b69c-4768-bad0-7d8925d0f835 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:43.930995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae82630-b69c-4768-bad0-7d8925d0f835 None admin] PATH_INFO: `/v3/projects/908a13d2b49544fbaf760b07e213bb7d/users/70ad51a8ad68440da6618891e2e08cb8/roles/336f27e7f8624a30b30cef9acada97a0` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:43.947566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae82630-b69c-4768-bad0-7d8925d0f835 None admin] RBAC: Authorizing `identity:create_grant(project_id=908a13d2b49544fbaf760b07e213bb7d, user_id=70ad51a8ad68440da6618891e2e08cb8, role_id=336f27e7f8624a30b30cef9acada97a0)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:43.949019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae82630-b69c-4768-bad0-7d8925d0f835 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:43.974137 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 305/610] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:41:43 2022] PUT /identity/v3/projects/908a13d2b49544fbaf760b07e213bb7d/users/70ad51a8ad68440da6618891e2e08cb8/roles/336f27e7f8624a30b30cef9acada97a0 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:43.974137 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae82630-b69c-4768-bad0-7d8925d0f835 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:44.035100 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b7f61-48ce-44ff-90d8-a45c59113e97 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:44.050424 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:44.077976 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7fdae9-70a3-4e13-a46b-eabb37b7897d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:44.079767 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fdae9-70a3-4e13-a46b-eabb37b7897d None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:44.080110 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fdae9-70a3-4e13-a46b-eabb37b7897d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:44.080441 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fdae9-70a3-4e13-a46b-eabb37b7897d None admin] PATH_INFO: `/v3/projects/908a13d2b49544fbaf760b07e213bb7d/users/50dbd3dd7e87481cb940a467c294006c/roles/336f27e7f8624a30b30cef9acada97a0` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:44.102523 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7fdae9-70a3-4e13-a46b-eabb37b7897d None admin] RBAC: Authorizing `identity:create_grant(project_id=908a13d2b49544fbaf760b07e213bb7d, user_id=50dbd3dd7e87481cb940a467c294006c, role_id=336f27e7f8624a30b30cef9acada97a0)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:44.104118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7fdae9-70a3-4e13-a46b-eabb37b7897d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:44.121138 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae82630-b69c-4768-bad0-7d8925d0f835 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:44.123572 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 306/611] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:41:43 2022] PUT /identity/v3/projects/908a13d2b49544fbaf760b07e213bb7d/users/50dbd3dd7e87481cb940a467c294006c/roles/336f27e7f8624a30b30cef9acada97a0 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:44.126451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b84cf5-1bd8-4040-93ab-56f6d6e49338 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:44.126793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b84cf5-1bd8-4040-93ab-56f6d6e49338 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:44.127079 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b84cf5-1bd8-4040-93ab-56f6d6e49338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:44.149178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-72b84cf5-1bd8-4040-93ab-56f6d6e49338 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:44.150658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-72b84cf5-1bd8-4040-93ab-56f6d6e49338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:44.153832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:44.220394 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98048ab-a97b-4530-a9b5-1ce52cd1746b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:44.221408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98048ab-a97b-4530-a9b5-1ce52cd1746b None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:44.222182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98048ab-a97b-4530-a9b5-1ce52cd1746b None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:44.222460 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98048ab-a97b-4530-a9b5-1ce52cd1746b None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:44.223321 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98048ab-a97b-4530-a9b5-1ce52cd1746b None admin] RBAC: Adding query filter params (name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:44.223627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98048ab-a97b-4530-a9b5-1ce52cd1746b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:44.225224 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98048ab-a97b-4530-a9b5-1ce52cd1746b None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:44.243859 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 306/612] 10.222.0.78 () {56 vars in 1103 bytes} [Wed May 25 14:41:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:44.249588 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98048ab-a97b-4530-a9b5-1ce52cd1746b None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:44.258118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 307/613] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:44.292348 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b84cf5-1bd8-4040-93ab-56f6d6e49338 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:44.381631 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98048ab-a97b-4530-a9b5-1ce52cd1746b None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:44.390296 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:44.408736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b84cf5-1bd8-4040-93ab-56f6d6e49338 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:44.424786 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a57d7f-fef5-49f3-9243-01ddc4ae1527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:44.425595 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a57d7f-fef5-49f3-9243-01ddc4ae1527 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:44.425873 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a57d7f-fef5-49f3-9243-01ddc4ae1527 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:44.426176 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a57d7f-fef5-49f3-9243-01ddc4ae1527 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:44.439917 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a57d7f-fef5-49f3-9243-01ddc4ae1527 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:44.441389 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a57d7f-fef5-49f3-9243-01ddc4ae1527 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:44.449481 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad9b61-7a5e-41d0-ae18-7eaef40b2554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:44.450330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad9b61-7a5e-41d0-ae18-7eaef40b2554 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:44.450618 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad9b61-7a5e-41d0-ae18-7eaef40b2554 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:44.450936 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad9b61-7a5e-41d0-ae18-7eaef40b2554 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:44.459404 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:44.463754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-25a57d7f-fef5-49f3-9243-01ddc4ae1527 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:44.463754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:44.463754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:44.463754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:44.463754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:44.463754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:44.463754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:44.463754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:44.463754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:44.463754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:44.463754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:44.463754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:44.463754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:44.463754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:44.463754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:44.465808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:44.465808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:44.465808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:44.465808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:44.465808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:44.465808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:44.465808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:44.465808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:44.465808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:44.465808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:44.465808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:44.465808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:44.465808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:44.465808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:44.465808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:44.465808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:44.465808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:44.467983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:44.467983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:44.467983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:44.467983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:44.468480 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abad9b61-7a5e-41d0-ae18-7eaef40b2554 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:44.474035 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 307/614] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:41:44.474035 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a57d7f-fef5-49f3-9243-01ddc4ae1527 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:44.474673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-abad9b61-7a5e-41d0-ae18-7eaef40b2554 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:44.474673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:44.474673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:44.474673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:44.474673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:44.474673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:44.474673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:44.474673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:44.474673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:44.474673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:44.474673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:44.474673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:44.474673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:44.474673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:44.474673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:44.474673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:44.476747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:44.476747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:44.476747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:44.476747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:44.476747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:44.476747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:44.476747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:44.476747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:44.476747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:44.476747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:44.476747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:44.476747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:44.476747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:44.476747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:44.476747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:44.476747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:44.478807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:44.478807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:44.478807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:44.478807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:44.478807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:44.478807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:44.478807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:44.478807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:44.478807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:44.478807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:44.478807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:44.478807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:44.478807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 308/615] 10.222.0.78 () {56 vars in 1102 bytes} [Wed May 25 14:41:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:44.486293 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad9b61-7a5e-41d0-ae18-7eaef40b2554 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:44.580229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a57d7f-fef5-49f3-9243-01ddc4ae1527 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:44.604946 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad9b61-7a5e-41d0-ae18-7eaef40b2554 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:44.617777 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39dbb54-0e4f-47b7-bb1f-9970a11d01a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:44.618538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dbb54-0e4f-47b7-bb1f-9970a11d01a1 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:44.618818 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dbb54-0e4f-47b7-bb1f-9970a11d01a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:44.619142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dbb54-0e4f-47b7-bb1f-9970a11d01a1 None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:44.619877 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39dbb54-0e4f-47b7-bb1f-9970a11d01a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:44.620182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39dbb54-0e4f-47b7-bb1f-9970a11d01a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:44.621560 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39dbb54-0e4f-47b7-bb1f-9970a11d01a1 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:44.640232 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 308/616] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:44.642220 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39dbb54-0e4f-47b7-bb1f-9970a11d01a1 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:44.647093 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3fb57-c6e2-4e9e-986c-4632cd0db7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:44.647808 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3fb57-c6e2-4e9e-986c-4632cd0db7a9 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:44.648093 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3fb57-c6e2-4e9e-986c-4632cd0db7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:44.648374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3fb57-c6e2-4e9e-986c-4632cd0db7a9 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:44.649164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3fb57-c6e2-4e9e-986c-4632cd0db7a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:44.649476 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3fb57-c6e2-4e9e-986c-4632cd0db7a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:44.650965 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3fb57-c6e2-4e9e-986c-4632cd0db7a9 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:44.660780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 309/617] 10.222.0.78 () {56 vars in 1103 bytes} [Wed May 25 14:41:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:44.666670 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3fb57-c6e2-4e9e-986c-4632cd0db7a9 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:44.670396 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:44.734757 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:44.785117 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3fb57-c6e2-4e9e-986c-4632cd0db7a9 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:44.791674 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39dbb54-0e4f-47b7-bb1f-9970a11d01a1 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:44.860473 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba873099-4c3f-46c6-b5ee-aca73bd73166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:44.861715 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba873099-4c3f-46c6-b5ee-aca73bd73166 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:44.862191 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba873099-4c3f-46c6-b5ee-aca73bd73166 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:44.862628 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba873099-4c3f-46c6-b5ee-aca73bd73166 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:44.878200 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba873099-4c3f-46c6-b5ee-aca73bd73166 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:44.879707 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba873099-4c3f-46c6-b5ee-aca73bd73166 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:44.882222 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718882df-9143-4bb6-9743-cc3d4ff6d70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:44.883041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718882df-9143-4bb6-9743-cc3d4ff6d70b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:44.883321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718882df-9143-4bb6-9743-cc3d4ff6d70b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:44.883596 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718882df-9143-4bb6-9743-cc3d4ff6d70b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:44.912675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-ba873099-4c3f-46c6-b5ee-aca73bd73166 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:41:44.912675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:44.912675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:44.912675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:44.912675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:44.912675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:44.912675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:44.912675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:44.912675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:44.912675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:44.912675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:44.912675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:44.912675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:41:44.912675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:41:44.912675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:41:44.912675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:41:44.913770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:44.913770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:44.913770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:41:44.913770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:44.913770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:41:44.913770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:44.913770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:44.913770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:44.913770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:44.913770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:44.913770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:44.913770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:44.913770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:44.913770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:44.913770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:44.913770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:44.913770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:44.913770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:44.914856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:44.914856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:44.914856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:41:44.914856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:41:44.914856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:41:44.914856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:41:44.914856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:41:44.914856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:41:44.914856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:41:44.914856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:44.920525 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718882df-9143-4bb6-9743-cc3d4ff6d70b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:44.922184 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 309/618] 10.222.0.78 () {56 vars in 1093 bytes} [Wed May 25 14:41:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:44.923236 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718882df-9143-4bb6-9743-cc3d4ff6d70b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:44.923558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba873099-4c3f-46c6-b5ee-aca73bd73166 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:44.945421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-718882df-9143-4bb6-9743-cc3d4ff6d70b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:44.945421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:44.945421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:44.945421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:44.945421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:44.945421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:44.945421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:44.945421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:44.945421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:44.945421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:44.945421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:44.945421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:44.945421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:44.945421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:44.945421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:44.946421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:44.946421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:44.946421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:44.946421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:44.946421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:44.946421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:44.946421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:44.946421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:44.946421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:44.946421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:44.946421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:44.946421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:44.946421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:44.946421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:44.946421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:44.946421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:44.946421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:44.947316 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:44.947316 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:44.947316 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:44.947316 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:44.951809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 310/619] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:41:44.953457 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718882df-9143-4bb6-9743-cc3d4ff6d70b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:45.092305 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718882df-9143-4bb6-9743-cc3d4ff6d70b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:45.110438 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:45.119799 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:45.128224 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba873099-4c3f-46c6-b5ee-aca73bd73166 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:45.167777 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139e98f-ef92-444a-9bcc-44e327f41c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:45.168459 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139e98f-ef92-444a-9bcc-44e327f41c96 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:45.168735 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139e98f-ef92-444a-9bcc-44e327f41c96 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:45.169007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139e98f-ef92-444a-9bcc-44e327f41c96 None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:45.169996 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139e98f-ef92-444a-9bcc-44e327f41c96 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:45.170479 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139e98f-ef92-444a-9bcc-44e327f41c96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:45.172649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139e98f-ef92-444a-9bcc-44e327f41c96 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:45.174870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84098ef8-2c96-4775-bcf5-9321b49a0acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:45.175548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84098ef8-2c96-4775-bcf5-9321b49a0acb None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:45.175878 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84098ef8-2c96-4775-bcf5-9321b49a0acb None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:45.176164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84098ef8-2c96-4775-bcf5-9321b49a0acb None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:45.176898 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84098ef8-2c96-4775-bcf5-9321b49a0acb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:45.177208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84098ef8-2c96-4775-bcf5-9321b49a0acb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:45.186827 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84098ef8-2c96-4775-bcf5-9321b49a0acb None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:45.194131 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 310/620] 10.222.0.78 () {56 vars in 1133 bytes} [Wed May 25 14:41:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:45.197116 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139e98f-ef92-444a-9bcc-44e327f41c96 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:45.212887 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 311/621] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:45.230851 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84098ef8-2c96-4775-bcf5-9321b49a0acb None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:45.367804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139e98f-ef92-444a-9bcc-44e327f41c96 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:45.426053 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84098ef8-2c96-4775-bcf5-9321b49a0acb None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:45.460178 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224bdd4-b449-417f-82f3-c22c5bca8163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:45.461300 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224bdd4-b449-417f-82f3-c22c5bca8163 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:45.461795 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224bdd4-b449-417f-82f3-c22c5bca8163 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:45.462317 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224bdd4-b449-417f-82f3-c22c5bca8163 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:45.474385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:45.489390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1ac29-be66-45bd-a2ca-615806126771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:45.490881 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1ac29-be66-45bd-a2ca-615806126771 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:45.491375 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1ac29-be66-45bd-a2ca-615806126771 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:45.491893 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1ac29-be66-45bd-a2ca-615806126771 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:45.509433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c1ac29-be66-45bd-a2ca-615806126771 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:45.515812 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c1ac29-be66-45bd-a2ca-615806126771 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:45.540840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-66c1ac29-be66-45bd-a2ca-615806126771 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 14:41:45.540840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:45.540840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:45.540840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:45.540840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:45.540840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:45.540840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:45.540840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:45.540840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:45.540840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:45.540840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:45.540840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:45.540840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:41:45.540840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:41:45.540840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:41:45.540840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:41:45.541874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:45.541874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:45.541874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:41:45.541874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:45.541874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:41:45.541874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:45.541874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:45.541874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:45.541874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:45.541874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:45.541874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:45.541874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:45.541874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:45.541874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:45.541874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:45.541874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:45.541874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:45.541874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:45.542832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:45.542832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:45.542832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:41:45.542832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:41:45.542832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:41:45.542832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:41:45.542832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:41:45.542832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:41:45.542832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 14:41:45.542832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:45.555156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 312/622] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:41:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:45.563583 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1ac29-be66-45bd-a2ca-615806126771 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:45.622811 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:45.625207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224bdd4-b449-417f-82f3-c22c5bca8163 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:45.627605 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224bdd4-b449-417f-82f3-c22c5bca8163 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:45.735679 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1ac29-be66-45bd-a2ca-615806126771 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:45.823653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 311/623] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:41:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 14:41:45.829725 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224bdd4-b449-417f-82f3-c22c5bca8163 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:45.834074 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037ae36-87aa-4266-9415-4d950284390e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:45.835159 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037ae36-87aa-4266-9415-4d950284390e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:45.835625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037ae36-87aa-4266-9415-4d950284390e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:45.836126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037ae36-87aa-4266-9415-4d950284390e None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:45.837151 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a037ae36-87aa-4266-9415-4d950284390e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:45.837611 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a037ae36-87aa-4266-9415-4d950284390e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:45.846822 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a037ae36-87aa-4266-9415-4d950284390e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:45.882400 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 313/624] 10.222.0.78 () {56 vars in 1135 bytes} [Wed May 25 14:41:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:45.888256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037ae36-87aa-4266-9415-4d950284390e None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:45.890949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:46.019996 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224bdd4-b449-417f-82f3-c22c5bca8163 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:46.046661 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:46.062966 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037ae36-87aa-4266-9415-4d950284390e None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:46.098924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6789458-3057-439a-828e-83aa8dc123bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:46.100093 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6789458-3057-439a-828e-83aa8dc123bc None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:46.103257 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6789458-3057-439a-828e-83aa8dc123bc None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:46.103763 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6789458-3057-439a-828e-83aa8dc123bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:46.127084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6789458-3057-439a-828e-83aa8dc123bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:46.129429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6789458-3057-439a-828e-83aa8dc123bc None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:46.134907 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d23d6-bf47-4637-af53-aeaa0416243a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:46.136000 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d23d6-bf47-4637-af53-aeaa0416243a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:46.137241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d23d6-bf47-4637-af53-aeaa0416243a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:46.138331 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d23d6-bf47-4637-af53-aeaa0416243a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:46.156181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d23d6-bf47-4637-af53-aeaa0416243a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:46.158178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d23d6-bf47-4637-af53-aeaa0416243a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:46.180577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-d6789458-3057-439a-828e-83aa8dc123bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:46.180577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:46.180577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:46.180577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:46.180577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:46.180577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:46.180577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:46.180577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:46.180577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:46.180577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:46.180577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:46.180577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:46.180577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:46.180577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:46.180577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:46.181527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:46.181527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:46.181527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:46.181527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:46.181527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:46.181527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:46.181527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:46.181527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:46.181527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:46.181527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:46.181527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:46.181527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:46.181527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:46.181527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:46.181527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:46.181527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:46.181527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:46.182424 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:46.182424 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:46.182424 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:46.182424 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:46.184841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 312/625] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:46.192730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6789458-3057-439a-828e-83aa8dc123bc None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:46.205240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-ca5d23d6-bf47-4637-af53-aeaa0416243a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:46.205240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:46.205240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:46.205240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:46.205240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:46.205240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:46.205240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:46.205240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:46.205240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:46.205240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:46.205240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:46.205240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:46.205240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:46.205240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:46.205240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:46.206224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:46.206224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:46.206224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:46.206224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:46.206224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:46.206224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:46.206224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:46.206224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:46.206224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:46.206224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:46.206224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:46.206224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:46.206224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:46.206224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:46.206224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:46.206224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:46.206224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:46.207082 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:46.207082 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:46.207082 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:46.207082 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:46.210236 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 314/626] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:41:46.218592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d23d6-bf47-4637-af53-aeaa0416243a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:46.393352 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d23d6-bf47-4637-af53-aeaa0416243a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:46.405710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:46.437233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6789458-3057-439a-828e-83aa8dc123bc None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:46.451394 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211901df-d79c-4217-b54e-abc5981ab201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:46.452681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211901df-d79c-4217-b54e-abc5981ab201 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:46.453105 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211901df-d79c-4217-b54e-abc5981ab201 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:46.453517 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211901df-d79c-4217-b54e-abc5981ab201 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:46.454447 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211901df-d79c-4217-b54e-abc5981ab201 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:46.454893 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211901df-d79c-4217-b54e-abc5981ab201 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:46.456419 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211901df-d79c-4217-b54e-abc5981ab201 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:46.474711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 315/627] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:46.482542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211901df-d79c-4217-b54e-abc5981ab201 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:46.490608 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a87d44-99e9-4fa8-8069-22244fb3693a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:46.492204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a87d44-99e9-4fa8-8069-22244fb3693a None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:46.492802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a87d44-99e9-4fa8-8069-22244fb3693a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:46.493335 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a87d44-99e9-4fa8-8069-22244fb3693a None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:46.494503 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a87d44-99e9-4fa8-8069-22244fb3693a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:46.495284 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a87d44-99e9-4fa8-8069-22244fb3693a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:46.496866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a87d44-99e9-4fa8-8069-22244fb3693a None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:46.498368 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:46.511033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 313/628] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:46.520221 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a87d44-99e9-4fa8-8069-22244fb3693a None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:46.666075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211901df-d79c-4217-b54e-abc5981ab201 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:46.711798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d6929e-dfca-4bc2-81a5-8fda812a2826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:46.713018 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6929e-dfca-4bc2-81a5-8fda812a2826 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:46.713500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6929e-dfca-4bc2-81a5-8fda812a2826 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:46.714122 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6929e-dfca-4bc2-81a5-8fda812a2826 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:46.726496 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d6929e-dfca-4bc2-81a5-8fda812a2826 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:46.728296 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d6929e-dfca-4bc2-81a5-8fda812a2826 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:46.753706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-c7d6929e-dfca-4bc2-81a5-8fda812a2826 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:46.753706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:46.753706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:46.753706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:46.753706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:46.753706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:46.753706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:46.753706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:46.753706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:46.753706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:46.753706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:46.753706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:46.753706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:41:46.753706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:41:46.753706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:41:46.754698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:41:46.754698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:46.754698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:46.754698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:46.754698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:46.754698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:46.754698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:46.754698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:46.754698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:46.754698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:46.754698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:46.754698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:46.754698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:46.754698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:46.754698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:46.754698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:46.754698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:46.755587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:41:46.755587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:41:46.755587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:41:46.755587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:41:46.755587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:41:46.755587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:41:46.755587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:46.755587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:46.758158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 316/629] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:41:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:41:46.759896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1415c4-8328-4909-a233-f0ed7cbc43c1 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:46.760400 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1415c4-8328-4909-a233-f0ed7cbc43c1 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:46.760873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1415c4-8328-4909-a233-f0ed7cbc43c1 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:46.762403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 317/630] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:46.763661 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e5bf4-d167-4b72-9069-1f1f395bb160 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:46.764208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e5bf4-d167-4b72-9069-1f1f395bb160 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:46.764706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e5bf4-d167-4b72-9069-1f1f395bb160 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:46.766363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 318/631] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:46.768251 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed1782-ded3-4663-977d-580f8172fb74 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:46.768577 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed1782-ded3-4663-977d-580f8172fb74 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:46.768907 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed1782-ded3-4663-977d-580f8172fb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:46.782897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a87d44-99e9-4fa8-8069-22244fb3693a None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:46.787279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b0ed1782-ded3-4663-977d-580f8172fb74 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:46.788464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b0ed1782-ded3-4663-977d-580f8172fb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:46.827378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b6e06-9a84-4633-ae2a-38f64834c018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:46.828199 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b6e06-9a84-4633-ae2a-38f64834c018 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:46.828474 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b6e06-9a84-4633-ae2a-38f64834c018 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:46.828750 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b6e06-9a84-4633-ae2a-38f64834c018 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:46.837186 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704b6e06-9a84-4633-ae2a-38f64834c018 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:46.838696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704b6e06-9a84-4633-ae2a-38f64834c018 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:46.852477 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:46.853353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-704b6e06-9a84-4633-ae2a-38f64834c018 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:46.853353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:46.853353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:46.853353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:46.853353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:46.853353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:46.853353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:46.853353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:46.853353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:46.853353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:46.853353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:46.853353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:46.853353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:41:46.853353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:41:46.853353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:41:46.854315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:41:46.854315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:46.854315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:46.854315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:46.854315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:46.854315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:46.854315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:46.854315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:46.854315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:46.854315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:46.854315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:46.854315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:46.854315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:46.854315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:46.854315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:46.854315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:46.854315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:46.855422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:41:46.855422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:41:46.855422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:41:46.855422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:41:46.855422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:41:46.855422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:41:46.855422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:46.855422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:46.855422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 314/632] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:41:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:46.855422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b6e06-9a84-4633-ae2a-38f64834c018 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:46.866400 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:46.868153 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 319/633] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:41:46.869484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b971a8-9e3c-46e2-82f7-c0ed2bd3b513 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:46.869773 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b971a8-9e3c-46e2-82f7-c0ed2bd3b513 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:46.870076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b971a8-9e3c-46e2-82f7-c0ed2bd3b513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:46.887459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-53b971a8-9e3c-46e2-82f7-c0ed2bd3b513 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:46.888640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-53b971a8-9e3c-46e2-82f7-c0ed2bd3b513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:46.959121 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 320/634] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:46.960476 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97ca30-82a0-4879-94e4-8ca913580572 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:46.960948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97ca30-82a0-4879-94e4-8ca913580572 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:46.961055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97ca30-82a0-4879-94e4-8ca913580572 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:46.962480 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 321/635] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:46.963207 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97ca30-82a0-4879-94e4-8ca913580572 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:46.975664 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b6e06-9a84-4633-ae2a-38f64834c018 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:47.011749 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b7e1f-214b-4124-b187-68225408ce91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:47.012418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b7e1f-214b-4124-b187-68225408ce91 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:47.012686 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b7e1f-214b-4124-b187-68225408ce91 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:47.012966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b7e1f-214b-4124-b187-68225408ce91 None admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:47.013751 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b7e1f-214b-4124-b187-68225408ce91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:47.014081 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b7e1f-214b-4124-b187-68225408ce91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:47.016115 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b7e1f-214b-4124-b187-68225408ce91 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:47.034486 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 315/636] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:41:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:47.034486 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b7e1f-214b-4124-b187-68225408ce91 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:47.080119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97ca30-82a0-4879-94e4-8ca913580572 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:47.098826 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:47.104781 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:47.117304 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0ee4d-d15c-4af6-8840-38024a47e899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:47.117971 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0ee4d-d15c-4af6-8840-38024a47e899 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:47.118305 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0ee4d-d15c-4af6-8840-38024a47e899 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:47.118556 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0ee4d-d15c-4af6-8840-38024a47e899 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:47.119337 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0ee4d-d15c-4af6-8840-38024a47e899 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:47.119644 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0ee4d-d15c-4af6-8840-38024a47e899 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:47.121081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0ee4d-d15c-4af6-8840-38024a47e899 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:47.133000 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 322/637] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:41:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:47.133568 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b7e1f-214b-4124-b187-68225408ce91 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:47.134406 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848a5e3-10af-4487-80d9-7397e57baf3a None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:47.134702 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848a5e3-10af-4487-80d9-7397e57baf3a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:47.134991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848a5e3-10af-4487-80d9-7397e57baf3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:47.152943 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-d848a5e3-10af-4487-80d9-7397e57baf3a None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:47.154212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d848a5e3-10af-4487-80d9-7397e57baf3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:47.176415 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121882e0-b600-4ff5-acbe-f3089e790417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:47.177205 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121882e0-b600-4ff5-acbe-f3089e790417 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:47.177480 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121882e0-b600-4ff5-acbe-f3089e790417 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:47.177748 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121882e0-b600-4ff5-acbe-f3089e790417 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:47.188330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121882e0-b600-4ff5-acbe-f3089e790417 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:47.195998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-121882e0-b600-4ff5-acbe-f3089e790417 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:47.195998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:47.195998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:47.195998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:47.195998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:47.195998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:47.195998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:47.195998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:47.195998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:47.195998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:47.195998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:47.195998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:47.195998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:47.195998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:47.195998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:47.195998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:47.197007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:47.197007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:47.197007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:47.197007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:47.197007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:47.197007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:47.197007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:47.197007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:47.197007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:47.197007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:47.197007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:47.197007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:47.197007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:47.197007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:47.197007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:47.197007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:47.197841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:47.197841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:47.197841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:47.197841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:47.197841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:47.197841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:47.197841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:47.197841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:47.197841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:47.197841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:47.197841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:47.197841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:47.197841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 316/638] 10.222.0.78 () {56 vars in 1102 bytes} [Wed May 25 14:41:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:47.198566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121882e0-b600-4ff5-acbe-f3089e790417 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:47.227875 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 323/639] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:47.228705 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848a5e3-10af-4487-80d9-7397e57baf3a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:47.332333 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121882e0-b600-4ff5-acbe-f3089e790417 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:47.363426 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848a5e3-10af-4487-80d9-7397e57baf3a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:47.373890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:47.380503 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:47.384907 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8949d9-9242-4c4e-9afc-1b02da865036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:47.385535 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8949d9-9242-4c4e-9afc-1b02da865036 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:47.385691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8949d9-9242-4c4e-9afc-1b02da865036 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:47.385963 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8949d9-9242-4c4e-9afc-1b02da865036 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:47.399000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8949d9-9242-4c4e-9afc-1b02da865036 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:47.407574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-2f8949d9-9242-4c4e-9afc-1b02da865036 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:47.407574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:47.407574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:47.407574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:47.407574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:47.407574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:47.407574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:47.407574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:47.407574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:47.407574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:47.407574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:47.407574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:47.407574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:47.407574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:47.407574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:47.407574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:47.408702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:47.408702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:47.408702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:47.408702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:47.408702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:47.408702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:47.408702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:47.408702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:47.408702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:47.408702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:47.408702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:47.408702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:47.408702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:47.408702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:47.408702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:47.408702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:47.409558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:47.409558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:47.409558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:47.409558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:47.409558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:47.409558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:47.409558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:47.409558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:47.409558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:47.409558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:47.409558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:47.409558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:47.410688 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f9372-1c4f-4e46-a95c-6536a680d179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:47.410795 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 317/640] 10.222.0.78 () {56 vars in 1102 bytes} [Wed May 25 14:41:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:47.410996 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f9372-1c4f-4e46-a95c-6536a680d179 None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:47.410996 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f9372-1c4f-4e46-a95c-6536a680d179 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:47.410996 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f9372-1c4f-4e46-a95c-6536a680d179 None admin] PATH_INFO: `/v3/projects/908a13d2b49544fbaf760b07e213bb7d/users/fbe4a2d106d84fc9ac65c046ba675353/roles/336f27e7f8624a30b30cef9acada97a0` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:47.411530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8949d9-9242-4c4e-9afc-1b02da865036 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:47.436255 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f9372-1c4f-4e46-a95c-6536a680d179 None admin] RBAC: Authorizing `identity:create_grant(project_id=908a13d2b49544fbaf760b07e213bb7d, user_id=fbe4a2d106d84fc9ac65c046ba675353, role_id=336f27e7f8624a30b30cef9acada97a0)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:47.437813 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f9372-1c4f-4e46-a95c-6536a680d179 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:47.459610 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 324/641] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:41:47 2022] PUT /identity/v3/projects/908a13d2b49544fbaf760b07e213bb7d/users/fbe4a2d106d84fc9ac65c046ba675353/roles/336f27e7f8624a30b30cef9acada97a0 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:47.460539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f9372-1c4f-4e46-a95c-6536a680d179 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:47.587115 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8949d9-9242-4c4e-9afc-1b02da865036 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:47.644496 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f9372-1c4f-4e46-a95c-6536a680d179 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:47.648345 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b8c55-2de1-4ec6-84ea-885529b8efb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:47.649058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b8c55-2de1-4ec6-84ea-885529b8efb5 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:47.649340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b8c55-2de1-4ec6-84ea-885529b8efb5 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:47.649613 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b8c55-2de1-4ec6-84ea-885529b8efb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:47.650560 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b8c55-2de1-4ec6-84ea-885529b8efb5 None admin] RBAC: Adding query filter params (role.id=336f27e7f8624a30b30cef9acada97a0, scope.project.id=908a13d2b49544fbaf760b07e213bb7d, user.id=e34e1d7ffbac49b0bde45829af0c6925) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:47.650885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b8c55-2de1-4ec6-84ea-885529b8efb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:47.652579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b8c55-2de1-4ec6-84ea-885529b8efb5 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:47.661026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 318/642] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:41:47 2022] GET /identity/v3/role_assignments?user.id=e34e1d7ffbac49b0bde45829af0c6925&scope.project.id=908a13d2b49544fbaf760b07e213bb7d&role.id=336f27e7f8624a30b30cef9acada97a0 => generated 608 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:47.663782 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b8c55-2de1-4ec6-84ea-885529b8efb5 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:47.712887 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e33b09-046a-4eee-966c-adf347a849c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:47.713606 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e33b09-046a-4eee-966c-adf347a849c9 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:47.713889 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e33b09-046a-4eee-966c-adf347a849c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:47.715354 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e33b09-046a-4eee-966c-adf347a849c9 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:47.716177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e33b09-046a-4eee-966c-adf347a849c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:47.716399 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e33b09-046a-4eee-966c-adf347a849c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:47.717922 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e33b09-046a-4eee-966c-adf347a849c9 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:47.729875 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 325/643] 10.222.0.78 () {56 vars in 1103 bytes} [Wed May 25 14:41:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:47.734922 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e33b09-046a-4eee-966c-adf347a849c9 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:47.781482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:47.784198 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:47.853212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b8c55-2de1-4ec6-84ea-885529b8efb5 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:47.912394 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e945cf1-d4d2-4dc5-baa9-aab12daa8c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:47.913259 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e945cf1-d4d2-4dc5-baa9-aab12daa8c4d None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:47.913544 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e945cf1-d4d2-4dc5-baa9-aab12daa8c4d None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:47.913819 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e945cf1-d4d2-4dc5-baa9-aab12daa8c4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:47.926855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e945cf1-d4d2-4dc5-baa9-aab12daa8c4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:47.931477 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e33b09-046a-4eee-966c-adf347a849c9 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:47.935729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-8e945cf1-d4d2-4dc5-baa9-aab12daa8c4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:47.935729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:47.935729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:47.935729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:47.935729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:47.935729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:47.935729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:47.935729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:47.935729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:47.935729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:47.935729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:47.935729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:47.935729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:47.935729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:47.935729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:47.935729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:47.937465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:47.937465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:47.937465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:47.937465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:47.937465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:47.937465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:47.937465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:47.937465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:47.937465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:47.937465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:47.937465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:47.937465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:47.937465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:47.937465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:47.937465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:47.937465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:47.938850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:47.938850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:47.938850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:47.938850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:47.938850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:47.938850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:47.938850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:47.938850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:47.938850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:47.938850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:47.938850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:47.938850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:47.939653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 319/644] 10.222.0.78 () {56 vars in 1102 bytes} [Wed May 25 14:41:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:47.940067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e945cf1-d4d2-4dc5-baa9-aab12daa8c4d None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:47.988010 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf5556-b8b9-4ad2-8015-8be7d16209ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:47.988743 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf5556-b8b9-4ad2-8015-8be7d16209ec None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:47.989034 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf5556-b8b9-4ad2-8015-8be7d16209ec None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:47.989315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf5556-b8b9-4ad2-8015-8be7d16209ec None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:47.991608 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf5556-b8b9-4ad2-8015-8be7d16209ec None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:47.991608 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf5556-b8b9-4ad2-8015-8be7d16209ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:47.991782 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf5556-b8b9-4ad2-8015-8be7d16209ec None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:48.007743 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 326/645] 10.222.0.78 () {56 vars in 1103 bytes} [Wed May 25 14:41:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:48.007743 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf5556-b8b9-4ad2-8015-8be7d16209ec None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:48.100273 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e945cf1-d4d2-4dc5-baa9-aab12daa8c4d None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:48.120314 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:48.120485 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:48.138523 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf5556-b8b9-4ad2-8015-8be7d16209ec None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:48.143042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637f4cd-8838-4549-a3bb-a5e9543daf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:48.143850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637f4cd-8838-4549-a3bb-a5e9543daf5e None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:48.144140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637f4cd-8838-4549-a3bb-a5e9543daf5e None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:48.144415 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637f4cd-8838-4549-a3bb-a5e9543daf5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:48.157085 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2637f4cd-8838-4549-a3bb-a5e9543daf5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:48.158650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2637f4cd-8838-4549-a3bb-a5e9543daf5e None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:48.179139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-2637f4cd-8838-4549-a3bb-a5e9543daf5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:48.179139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:48.179139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:48.179139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:48.179139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:48.179139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:48.179139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:48.179139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:48.179139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:48.179139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:48.179139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:48.179139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:48.179139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:48.179139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:48.179139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:48.180084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:48.180084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:48.180084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:48.180084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:48.180084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:48.180084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:48.180084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:48.180084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:48.180084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:48.180084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:48.180084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:48.180084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:48.180084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:48.180084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:48.180084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:48.180084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:48.180084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:48.181114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:48.181114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:48.181114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:48.181114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:48.181366 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ed38e-cc71-43ab-8ddc-3f785787d90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:48.181366 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ed38e-cc71-43ab-8ddc-3f785787d90f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:48.181366 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ed38e-cc71-43ab-8ddc-3f785787d90f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:48.181366 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ed38e-cc71-43ab-8ddc-3f785787d90f None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:48.181626 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 320/646] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:48.181700 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ed38e-cc71-43ab-8ddc-3f785787d90f None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:48.181777 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637f4cd-8838-4549-a3bb-a5e9543daf5e None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:48.181853 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ed38e-cc71-43ab-8ddc-3f785787d90f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:48.183024 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ed38e-cc71-43ab-8ddc-3f785787d90f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:48.192750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 327/647] 10.222.0.78 () {56 vars in 1103 bytes} [Wed May 25 14:41:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:48.193038 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ed38e-cc71-43ab-8ddc-3f785787d90f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:48.318923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637f4cd-8838-4549-a3bb-a5e9543daf5e None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:48.319226 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ed38e-cc71-43ab-8ddc-3f785787d90f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:48.356403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab6247-d292-49cc-9cf3-c3a89ef5e5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:48.357103 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab6247-d292-49cc-9cf3-c3a89ef5e5e4 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:48.357386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab6247-d292-49cc-9cf3-c3a89ef5e5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:48.357680 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab6247-d292-49cc-9cf3-c3a89ef5e5e4 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:48.359262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab6247-d292-49cc-9cf3-c3a89ef5e5e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:48.359569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab6247-d292-49cc-9cf3-c3a89ef5e5e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:48.360852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f3759-12ec-410f-b643-b6ef853a348a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:48.360995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab6247-d292-49cc-9cf3-c3a89ef5e5e4 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:48.362045 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f3759-12ec-410f-b643-b6ef853a348a None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:48.362651 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f3759-12ec-410f-b643-b6ef853a348a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:48.363135 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f3759-12ec-410f-b643-b6ef853a348a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:48.376031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f3759-12ec-410f-b643-b6ef853a348a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:48.376612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 328/648] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:48.377564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab6247-d292-49cc-9cf3-c3a89ef5e5e4 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:48.378526 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f3759-12ec-410f-b643-b6ef853a348a None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:48.392951 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:48.408019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-ef9f3759-12ec-410f-b643-b6ef853a348a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:48.408019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:48.408019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:48.408019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:48.408019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:48.408019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:48.408019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:48.408019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:48.408019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:48.408019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:48.408019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:48.408019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:48.408019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:48.408019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:48.408019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:48.409038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:48.409038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:48.409038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:48.409038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:48.409038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:48.409038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:48.409038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:48.409038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:48.409038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:48.409038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:48.409038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:48.409038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:48.409038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:48.409038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:48.409038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:48.409038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:48.409038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:48.409980 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:48.409980 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:48.409980 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:48.409980 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:48.412545 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 321/649] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:48.413656 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f3759-12ec-410f-b643-b6ef853a348a None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:48.433800 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:48.510873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab6247-d292-49cc-9cf3-c3a89ef5e5e4 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:48.545044 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f3759-12ec-410f-b643-b6ef853a348a None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:48.551935 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288186d0-292e-444f-8d5b-6a9fc1169d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:48.553076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288186d0-292e-444f-8d5b-6a9fc1169d3d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:48.553690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288186d0-292e-444f-8d5b-6a9fc1169d3d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:48.554262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288186d0-292e-444f-8d5b-6a9fc1169d3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:48.566917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288186d0-292e-444f-8d5b-6a9fc1169d3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:48.568720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288186d0-292e-444f-8d5b-6a9fc1169d3d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:48.586947 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e5664-3f87-4d23-975d-4cf08e3b5acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:48.587745 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e5664-3f87-4d23-975d-4cf08e3b5acb None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:48.588018 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e5664-3f87-4d23-975d-4cf08e3b5acb None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:48.588288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e5664-3f87-4d23-975d-4cf08e3b5acb None admin] PATH_INFO: `/v3/users/glance` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:48.590659 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-288186d0-292e-444f-8d5b-6a9fc1169d3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:48.590659 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:48.590659 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:48.590659 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:48.590659 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:48.590659 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:48.590659 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:48.590659 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:48.590659 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:48.590659 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:48.590659 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:48.590659 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:48.590659 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:48.590659 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:48.590659 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:48.591548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:48.591548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:48.591548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:48.591548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:48.591548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:48.591548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:48.591548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:48.591548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:48.591548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:48.591548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:48.591548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:48.591548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:48.591548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:48.591548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:48.591548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:48.591548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:48.591548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:48.593003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:48.593003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:48.593003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:48.593003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:48.599691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e5664-3f87-4d23-975d-4cf08e3b5acb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:48.599691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e5664-3f87-4d23-975d-4cf08e3b5acb None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:48.600239 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 329/650] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:48.601395 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288186d0-292e-444f-8d5b-6a9fc1169d3d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:48.620582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-0e0e5664-3f87-4d23-975d-4cf08e3b5acb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 25 14:41:48.620582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:48.620582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:48.620582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:48.620582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:48.620582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:48.620582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:48.620582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:48.620582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:48.620582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:48.620582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:48.620582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:48.620582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:41:48.620582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:41:48.620582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:41:48.620582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:41:48.621665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:48.621665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:48.621665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:41:48.621665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:48.621665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:41:48.621665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:48.621665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:48.621665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:48.621665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:48.621665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:48.621665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:48.621665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:48.621665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:48.621665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:48.621665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:48.621665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:48.621665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:48.621665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:48.622699 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:48.622699 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:48.622699 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:41:48.622699 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:41:48.622699 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:41:48.622699 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:41:48.622699 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:41:48.622699 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:41:48.622699 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 25 14:41:48.622699 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:48.622699 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 322/651] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:41:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:48.623324 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e5664-3f87-4d23-975d-4cf08e3b5acb None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:48.690363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:48.747465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:48.755216 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288186d0-292e-444f-8d5b-6a9fc1169d3d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:48.779283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e5664-3f87-4d23-975d-4cf08e3b5acb None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:48.796361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af62df-abfa-4f1c-b8d0-1adce0aca9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:48.797042 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af62df-abfa-4f1c-b8d0-1adce0aca9da None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:48.797329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af62df-abfa-4f1c-b8d0-1adce0aca9da None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:48.797610 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af62df-abfa-4f1c-b8d0-1adce0aca9da None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:48.798384 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af62df-abfa-4f1c-b8d0-1adce0aca9da None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:48.798695 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af62df-abfa-4f1c-b8d0-1adce0aca9da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:48.800264 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af62df-abfa-4f1c-b8d0-1adce0aca9da None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:48.813024 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 330/652] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:48.814151 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af62df-abfa-4f1c-b8d0-1adce0aca9da None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:48.829002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e8401-1705-44ac-a6c4-d3a830eaf34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:48.830596 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e8401-1705-44ac-a6c4-d3a830eaf34b None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:48.831249 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e8401-1705-44ac-a6c4-d3a830eaf34b None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:48.831714 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e8401-1705-44ac-a6c4-d3a830eaf34b None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:48.832875 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e8401-1705-44ac-a6c4-d3a830eaf34b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:48.833845 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e8401-1705-44ac-a6c4-d3a830eaf34b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:48.836486 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e8401-1705-44ac-a6c4-d3a830eaf34b None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:48.855725 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 323/653] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:48.858350 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e8401-1705-44ac-a6c4-d3a830eaf34b None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:48.943072 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af62df-abfa-4f1c-b8d0-1adce0aca9da None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:48.970458 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:48.992176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9747b4e-1ecb-4ae1-830e-30a1318b7326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:48.992851 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9747b4e-1ecb-4ae1-830e-30a1318b7326 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:48.993138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9747b4e-1ecb-4ae1-830e-30a1318b7326 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:48.993416 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9747b4e-1ecb-4ae1-830e-30a1318b7326 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:48.994292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9747b4e-1ecb-4ae1-830e-30a1318b7326 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:48.994604 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9747b4e-1ecb-4ae1-830e-30a1318b7326 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:48.996176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9747b4e-1ecb-4ae1-830e-30a1318b7326 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:48.996595 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e8401-1705-44ac-a6c4-d3a830eaf34b None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:49.016574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 331/654] 10.222.0.78 () {56 vars in 1137 bytes} [Wed May 25 14:41:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:49.017897 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9747b4e-1ecb-4ae1-830e-30a1318b7326 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:49.047556 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ee728-494f-4a58-b847-b4fbf0f8ce63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:49.049041 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ee728-494f-4a58-b847-b4fbf0f8ce63 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:49.050068 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ee728-494f-4a58-b847-b4fbf0f8ce63 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:49.050889 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ee728-494f-4a58-b847-b4fbf0f8ce63 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:49.055696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:49.063577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ee728-494f-4a58-b847-b4fbf0f8ce63 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:49.067670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ee728-494f-4a58-b847-b4fbf0f8ce63 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:49.090058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-c70ee728-494f-4a58-b847-b4fbf0f8ce63 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 25 14:41:49.090058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:49.090058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:49.090058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:49.090058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:49.090058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:49.090058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:49.090058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:49.090058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:49.090058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:49.090058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:49.090058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:49.090058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:41:49.090058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:41:49.090058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:41:49.090058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:41:49.091051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:49.091051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:49.091051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:41:49.091051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:49.091051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:41:49.091051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:49.091051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:49.091051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:49.091051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:49.091051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:49.091051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:49.091051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:49.091051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:49.091051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:49.091051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:49.091051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:49.091051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:49.091051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:49.091955 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:49.091955 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:49.091955 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:41:49.091955 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:41:49.091955 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:41:49.091955 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:41:49.091955 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:41:49.091955 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:41:49.091955 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 25 14:41:49.091955 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:49.095934 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 324/655] 10.222.0.78 () {56 vars in 1102 bytes} [Wed May 25 14:41:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:49.097817 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ee728-494f-4a58-b847-b4fbf0f8ce63 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:49.142308 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9747b4e-1ecb-4ae1-830e-30a1318b7326 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:49.186562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34f823-b3ef-4088-aa0a-ce481835de99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:49.187994 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34f823-b3ef-4088-aa0a-ce481835de99 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:49.188448 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34f823-b3ef-4088-aa0a-ce481835de99 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:49.189114 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34f823-b3ef-4088-aa0a-ce481835de99 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:49.200376 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a34f823-b3ef-4088-aa0a-ce481835de99 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:49.203155 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a34f823-b3ef-4088-aa0a-ce481835de99 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:49.223780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-5a34f823-b3ef-4088-aa0a-ce481835de99 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 14:41:49.223780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:49.223780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:49.223780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:49.223780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:49.223780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:49.223780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:49.223780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:49.223780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:49.223780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:49.223780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:49.223780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:49.223780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:41:49.223780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:41:49.223780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:41:49.223780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:41:49.224947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:49.224947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:49.224947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:41:49.224947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:49.224947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:41:49.224947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:49.224947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:49.224947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:49.224947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:49.224947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:49.224947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:49.224947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:49.224947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:49.224947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:49.224947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:49.224947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:49.224947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:49.224947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:49.225855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:49.225855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:49.225855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:41:49.225855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:41:49.225855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:41:49.225855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:41:49.225855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:41:49.225855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:41:49.225855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 14:41:49.225855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:49.230094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 332/656] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:41:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:49.231285 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34f823-b3ef-4088-aa0a-ce481835de99 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:49.247259 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ee728-494f-4a58-b847-b4fbf0f8ce63 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:49.289281 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:49.297730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b3ba0-26dc-45ee-a893-68e73a03c70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:49.298423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b3ba0-26dc-45ee-a893-68e73a03c70b None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:49.298698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b3ba0-26dc-45ee-a893-68e73a03c70b None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:49.298965 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b3ba0-26dc-45ee-a893-68e73a03c70b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:49.380953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34f823-b3ef-4088-aa0a-ce481835de99 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:49.390046 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:49.400940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b3ba0-26dc-45ee-a893-68e73a03c70b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:49.403659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b3ba0-26dc-45ee-a893-68e73a03c70b None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:49.425871 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364623e1-e150-48c5-bcb2-fbf0e2927a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:49.425871 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364623e1-e150-48c5-bcb2-fbf0e2927a1b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:49.425871 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364623e1-e150-48c5-bcb2-fbf0e2927a1b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:49.425871 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364623e1-e150-48c5-bcb2-fbf0e2927a1b None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:49.429336 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364623e1-e150-48c5-bcb2-fbf0e2927a1b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:49.429336 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364623e1-e150-48c5-bcb2-fbf0e2927a1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:49.429336 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364623e1-e150-48c5-bcb2-fbf0e2927a1b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:49.447945 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 333/657] 10.222.0.78 () {56 vars in 1139 bytes} [Wed May 25 14:41:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:49.448962 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364623e1-e150-48c5-bcb2-fbf0e2927a1b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:49.546087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 325/658] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:41:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:41:49.547803 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b3ba0-26dc-45ee-a893-68e73a03c70b None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:49.555047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:49.564094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364623e1-e150-48c5-bcb2-fbf0e2927a1b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:49.602402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f1eb8-c848-4916-831d-cf4bb9af2e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:49.603399 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f1eb8-c848-4916-831d-cf4bb9af2e89 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:49.603957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f1eb8-c848-4916-831d-cf4bb9af2e89 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:49.604433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f1eb8-c848-4916-831d-cf4bb9af2e89 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:49.605476 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f1eb8-c848-4916-831d-cf4bb9af2e89 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:49.605969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f1eb8-c848-4916-831d-cf4bb9af2e89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:49.607892 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f1eb8-c848-4916-831d-cf4bb9af2e89 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:49.629390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 334/659] 10.222.0.78 () {56 vars in 1137 bytes} [Wed May 25 14:41:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:49.630646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f1eb8-c848-4916-831d-cf4bb9af2e89 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:49.679439 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:49.688131 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b3ba0-26dc-45ee-a893-68e73a03c70b None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:49.727393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399e31c-375f-4511-8691-c1e5f44d96dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:49.728037 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399e31c-375f-4511-8691-c1e5f44d96dc None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:49.728312 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399e31c-375f-4511-8691-c1e5f44d96dc None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:49.728579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399e31c-375f-4511-8691-c1e5f44d96dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:49.773174 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f1eb8-c848-4916-831d-cf4bb9af2e89 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:49.803641 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1399e31c-375f-4511-8691-c1e5f44d96dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:49.805052 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1399e31c-375f-4511-8691-c1e5f44d96dc None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:49.824295 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106f8f2-0fe6-43c0-af92-433c176bf3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:49.826106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106f8f2-0fe6-43c0-af92-433c176bf3a3 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:49.827275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106f8f2-0fe6-43c0-af92-433c176bf3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:49.827789 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106f8f2-0fe6-43c0-af92-433c176bf3a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:49.842784 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106f8f2-0fe6-43c0-af92-433c176bf3a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:49.844365 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106f8f2-0fe6-43c0-af92-433c176bf3a3 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:49.864028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-0106f8f2-0fe6-43c0-af92-433c176bf3a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:49.864028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:49.864028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:49.864028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:49.864028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:49.864028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:49.864028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:49.864028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:49.864028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:49.864028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:49.864028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:49.864028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:49.864028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:49.864028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:49.864028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:49.865279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:49.865279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:49.865279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:49.865279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:49.865279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:49.865279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:49.865279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:49.865279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:49.865279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:49.865279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:49.865279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:49.865279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:49.865279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:49.865279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:49.865279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:49.865279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:49.865279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:49.866261 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:49.866261 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:49.866261 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:49.866261 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:49.866261 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 335/660] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:49.866993 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106f8f2-0fe6-43c0-af92-433c176bf3a3 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:49.878330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:49.935158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:49.949218 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 326/661] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:41:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:41:49.950821 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06240b95-310a-4900-9431-afcc2529c2d1 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:49.951103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06240b95-310a-4900-9431-afcc2529c2d1 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:49.951385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06240b95-310a-4900-9431-afcc2529c2d1 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:49.953209 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 327/662] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:49.953695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06240b95-310a-4900-9431-afcc2529c2d1 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:50.007334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106f8f2-0fe6-43c0-af92-433c176bf3a3 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:50.047704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2ef21-1826-49d6-8741-9e1ca8754d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:50.048368 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2ef21-1826-49d6-8741-9e1ca8754d08 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:50.048655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2ef21-1826-49d6-8741-9e1ca8754d08 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:50.048932 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2ef21-1826-49d6-8741-9e1ca8754d08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:50.080169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06240b95-310a-4900-9431-afcc2529c2d1 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:50.116111 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c842b-bda8-49e8-ba0a-a127c48be35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:50.116831 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c842b-bda8-49e8-ba0a-a127c48be35f None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:50.117113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c842b-bda8-49e8-ba0a-a127c48be35f None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:50.117390 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c842b-bda8-49e8-ba0a-a127c48be35f None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:50.118146 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c842b-bda8-49e8-ba0a-a127c48be35f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:50.118448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c842b-bda8-49e8-ba0a-a127c48be35f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:50.119883 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c842b-bda8-49e8-ba0a-a127c48be35f None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:50.128022 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f2ef21-1826-49d6-8741-9e1ca8754d08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:50.129505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f2ef21-1826-49d6-8741-9e1ca8754d08 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:50.132073 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 328/663] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:50.137164 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c842b-bda8-49e8-ba0a-a127c48be35f None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:50.138556 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:50.203418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:50.242844 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 336/664] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:41:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:41:50.244156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01019a0-3916-4504-a341-23f7777bd1ac None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:50.244447 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01019a0-3916-4504-a341-23f7777bd1ac None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:50.244726 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01019a0-3916-4504-a341-23f7777bd1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:50.253224 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c842b-bda8-49e8-ba0a-a127c48be35f None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:50.260782 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b01019a0-3916-4504-a341-23f7777bd1ac None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:50.261981 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b01019a0-3916-4504-a341-23f7777bd1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:50.288043 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2a464-c859-4d83-9703-a41ab278382f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:50.288826 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2a464-c859-4d83-9703-a41ab278382f None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:50.289106 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2a464-c859-4d83-9703-a41ab278382f None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:50.289377 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2a464-c859-4d83-9703-a41ab278382f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:50.300525 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2a464-c859-4d83-9703-a41ab278382f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:50.301965 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2a464-c859-4d83-9703-a41ab278382f None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:50.324095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-97f2a464-c859-4d83-9703-a41ab278382f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:50.324095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:50.324095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:50.324095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:50.324095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:50.324095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:50.324095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:50.324095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:50.324095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:50.324095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:50.324095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:50.324095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:50.324095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:50.324095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:50.324095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:50.325236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:50.325236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:50.325236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:50.325236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:50.325236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:50.325236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:50.325236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:50.325236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:50.325236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:50.325236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:50.325236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:50.325236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:50.325236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:50.325236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:50.325236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:50.325236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:50.325236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:50.326213 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:50.326213 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:50.326213 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:50.326213 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:50.326213 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 329/665] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:50.327281 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2a464-c859-4d83-9703-a41ab278382f None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:50.336139 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 337/666] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:50.337424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a715f71-54f9-4963-bbb5-bdd4a33dc215 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:50.337716 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a715f71-54f9-4963-bbb5-bdd4a33dc215 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:50.337994 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a715f71-54f9-4963-bbb5-bdd4a33dc215 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:50.339329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 338/667] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:50.340033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a715f71-54f9-4963-bbb5-bdd4a33dc215 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:50.380444 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:50.447344 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a715f71-54f9-4963-bbb5-bdd4a33dc215 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:50.449534 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2a464-c859-4d83-9703-a41ab278382f None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:50.466203 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:50.482576 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ade7d2-ae79-4aa6-9688-2057aa187cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:50.483533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ade7d2-ae79-4aa6-9688-2057aa187cb9 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:50.483815 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ade7d2-ae79-4aa6-9688-2057aa187cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:50.484091 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ade7d2-ae79-4aa6-9688-2057aa187cb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:50.494513 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ade7d2-ae79-4aa6-9688-2057aa187cb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:50.496100 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ade7d2-ae79-4aa6-9688-2057aa187cb9 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:50.496367 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559021d0-6f10-4790-98fc-b9a310889c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:50.497158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559021d0-6f10-4790-98fc-b9a310889c36 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:50.497438 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559021d0-6f10-4790-98fc-b9a310889c36 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:50.497713 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559021d0-6f10-4790-98fc-b9a310889c36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:50.505252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559021d0-6f10-4790-98fc-b9a310889c36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:50.506746 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559021d0-6f10-4790-98fc-b9a310889c36 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:50.515829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-a1ade7d2-ae79-4aa6-9688-2057aa187cb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:50.515829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:50.515829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:50.515829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:50.515829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:50.515829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:50.515829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:50.515829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:50.515829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:50.515829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:50.515829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:50.515829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:50.515829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:50.515829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:50.515829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:50.516722 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:50.516722 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:50.516722 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:50.516722 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:50.516722 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:50.516722 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:50.516722 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:50.516722 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:50.516722 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:50.516722 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:50.516722 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:50.516722 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:50.516722 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:50.516722 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:50.516722 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:50.516722 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:50.516722 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:50.517609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:50.517609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:50.517609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:50.517609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:50.517609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 339/668] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:50.517875 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ade7d2-ae79-4aa6-9688-2057aa187cb9 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:50.521627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-559021d0-6f10-4790-98fc-b9a310889c36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:50.521627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:50.521627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:50.521627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:50.521627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:50.521627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:50.521627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:50.521627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:50.521627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:50.521627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:50.521627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:50.521627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:50.521627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:41:50.521627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:41:50.521627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:41:50.522636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:41:50.522636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:50.522636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:50.522636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:50.522636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:50.522636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:50.522636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:50.522636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:50.522636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:50.522636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:50.522636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:50.522636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:50.522636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:50.522636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:50.522636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:50.522636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:50.522636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:50.523506 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:41:50.523506 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:41:50.523506 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:41:50.523506 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:41:50.523506 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:41:50.523506 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:41:50.523506 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:50.523506 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:50.523506 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 330/669] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:41:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:50.524540 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef97b3-b1ed-477b-b829-67315526dcf4 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:50.524540 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef97b3-b1ed-477b-b829-67315526dcf4 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:50.524685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef97b3-b1ed-477b-b829-67315526dcf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:50.543112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-20ef97b3-b1ed-477b-b829-67315526dcf4 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:50.544321 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-20ef97b3-b1ed-477b-b829-67315526dcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:50.617967 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:50.620577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 331/670] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:50.621497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef97b3-b1ed-477b-b829-67315526dcf4 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:50.630477 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ade7d2-ae79-4aa6-9688-2057aa187cb9 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:50.667134 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae327f23-d131-4049-8f3a-199d70454d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:50.667798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae327f23-d131-4049-8f3a-199d70454d09 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:50.668079 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae327f23-d131-4049-8f3a-199d70454d09 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:50.668357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae327f23-d131-4049-8f3a-199d70454d09 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:50.669066 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae327f23-d131-4049-8f3a-199d70454d09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:50.669370 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae327f23-d131-4049-8f3a-199d70454d09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:50.671395 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae327f23-d131-4049-8f3a-199d70454d09 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:50.683027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 340/671] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:50.683870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae327f23-d131-4049-8f3a-199d70454d09 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:50.744546 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:50.748238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef97b3-b1ed-477b-b829-67315526dcf4 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:50.785642 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9929bfa-386c-4a13-8bc9-b0d8abdfbadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:50.786734 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9929bfa-386c-4a13-8bc9-b0d8abdfbadf None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:50.787021 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9929bfa-386c-4a13-8bc9-b0d8abdfbadf None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:50.787291 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9929bfa-386c-4a13-8bc9-b0d8abdfbadf None admin] PATH_INFO: `/v3/roles/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:50.790723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae327f23-d131-4049-8f3a-199d70454d09 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:50.798721 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9929bfa-386c-4a13-8bc9-b0d8abdfbadf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:50.806504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-f9929bfa-386c-4a13-8bc9-b0d8abdfbadf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:50.806504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:50.806504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:50.806504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:50.806504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:50.806504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:50.806504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:50.806504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:50.806504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:50.806504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:50.806504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:50.806504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:50.806504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:50.806504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:50.806504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:50.806504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:50.807434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:50.807434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:50.807434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:50.807434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:50.807434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:50.807434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:50.807434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:50.807434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:50.807434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:50.807434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:50.807434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:50.807434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:50.807434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:50.807434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:50.807434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:50.807434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:50.808272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:50.808272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:50.808272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:50.808272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:50.808272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:50.808272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:50.808272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:50.808272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:50.808272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:50.808272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:50.808272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:50.808272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:50.808272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 332/672] 10.222.0.78 () {56 vars in 1102 bytes} [Wed May 25 14:41:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:50.808991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9929bfa-386c-4a13-8bc9-b0d8abdfbadf None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:50.829395 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e424a0-1bf9-459e-bd22-287a6cf503c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:50.830122 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e424a0-1bf9-459e-bd22-287a6cf503c7 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:50.830411 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e424a0-1bf9-459e-bd22-287a6cf503c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:50.830690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e424a0-1bf9-459e-bd22-287a6cf503c7 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:50.831423 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e424a0-1bf9-459e-bd22-287a6cf503c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:50.831736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e424a0-1bf9-459e-bd22-287a6cf503c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:50.833246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e424a0-1bf9-459e-bd22-287a6cf503c7 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:50.847870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 341/673] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:50.849304 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e424a0-1bf9-459e-bd22-287a6cf503c7 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:50.867316 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:50.933392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9929bfa-386c-4a13-8bc9-b0d8abdfbadf None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:50.961589 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e424a0-1bf9-459e-bd22-287a6cf503c7 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:50.969548 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04267d5-7e93-445e-8375-2766c9fdcd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:50.970237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04267d5-7e93-445e-8375-2766c9fdcd69 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:50.970517 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04267d5-7e93-445e-8375-2766c9fdcd69 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:50.970792 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04267d5-7e93-445e-8375-2766c9fdcd69 None admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:50.971580 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04267d5-7e93-445e-8375-2766c9fdcd69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:50.971890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04267d5-7e93-445e-8375-2766c9fdcd69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:50.973320 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04267d5-7e93-445e-8375-2766c9fdcd69 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:50.989588 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 333/674] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:41:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:50.990337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04267d5-7e93-445e-8375-2766c9fdcd69 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:51.003781 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3802e53d-3a6b-434f-b3db-00cf9d0732c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:51.004784 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3802e53d-3a6b-434f-b3db-00cf9d0732c4 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:51.005627 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3802e53d-3a6b-434f-b3db-00cf9d0732c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:51.005627 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3802e53d-3a6b-434f-b3db-00cf9d0732c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:51.011650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:51.018252 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3802e53d-3a6b-434f-b3db-00cf9d0732c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:51.026064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-3802e53d-3a6b-434f-b3db-00cf9d0732c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:41:51.026064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:51.026064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:51.026064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:51.026064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:51.026064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:51.026064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:51.026064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:51.026064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:51.026064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:51.026064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:51.026064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:51.026064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:51.026064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:51.026064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:51.026064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:51.027171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:51.027171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:51.027171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:51.027171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:51.027171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:51.027171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:51.027171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:51.027171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:51.027171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:51.027171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:51.027171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:51.027171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:51.027171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:51.027171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:51.027171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:51.027171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:51.028027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:51.028027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:51.028027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:51.028027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:51.028027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:51.028027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:51.028027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:51.028027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:51.028027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:51.028027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:51.028027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:41:51.028027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:51.028027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 342/675] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:41:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:51.028762 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3802e53d-3a6b-434f-b3db-00cf9d0732c4 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:51.099396 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:51.107123 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04267d5-7e93-445e-8375-2766c9fdcd69 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:51.134932 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3802e53d-3a6b-434f-b3db-00cf9d0732c4 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:51.143346 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19bfe80-730d-4503-88e3-fd46ba6d33c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:51.144141 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bfe80-730d-4503-88e3-fd46ba6d33c1 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:51.144699 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bfe80-730d-4503-88e3-fd46ba6d33c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:51.144699 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bfe80-730d-4503-88e3-fd46ba6d33c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:51.151822 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19bfe80-730d-4503-88e3-fd46ba6d33c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:51.153347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19bfe80-730d-4503-88e3-fd46ba6d33c1 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:51.170210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-b19bfe80-730d-4503-88e3-fd46ba6d33c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:51.170210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:51.170210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:51.170210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:51.170210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:51.170210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:51.170210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:51.170210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:51.170210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:51.170210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:51.170210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:51.170210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:51.170210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:41:51.170210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:41:51.170210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:41:51.172033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:41:51.172033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:51.172033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:51.172033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:51.172033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:51.172033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:51.172033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:51.172033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:51.172033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:51.172033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:51.172033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:51.172033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:51.172033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:51.172033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:51.172033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:51.172033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:51.172033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:51.173522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:41:51.173522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:41:51.173522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:41:51.173522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:41:51.173522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:41:51.173522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:41:51.173522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:51.173522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:51.173522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 334/676] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:41:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:51.174218 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7160f-3d04-4335-95bf-76bc2e9fad0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:51.174323 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19bfe80-730d-4503-88e3-fd46ba6d33c1 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:51.174407 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7160f-3d04-4335-95bf-76bc2e9fad0c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:51.174407 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7160f-3d04-4335-95bf-76bc2e9fad0c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:51.174407 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7160f-3d04-4335-95bf-76bc2e9fad0c None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:51.174407 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7160f-3d04-4335-95bf-76bc2e9fad0c None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:51.174697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7160f-3d04-4335-95bf-76bc2e9fad0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:51.175940 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7160f-3d04-4335-95bf-76bc2e9fad0c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:51.185613 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 343/677] 10.222.0.78 () {56 vars in 1103 bytes} [Wed May 25 14:41:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:51.186559 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7160f-3d04-4335-95bf-76bc2e9fad0c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:51.259188 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:51.283002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19bfe80-730d-4503-88e3-fd46ba6d33c1 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:51.293280 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7160f-3d04-4335-95bf-76bc2e9fad0c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:51.317912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5a20f-ff08-4ecb-9fd6-9475ac898de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:51.318770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5a20f-ff08-4ecb-9fd6-9475ac898de2 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:51.319049 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5a20f-ff08-4ecb-9fd6-9475ac898de2 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:51.319319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5a20f-ff08-4ecb-9fd6-9475ac898de2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:51.322515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:51.326367 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5a20f-ff08-4ecb-9fd6-9475ac898de2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:51.327852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5a20f-ff08-4ecb-9fd6-9475ac898de2 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:51.328654 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51679c48-4582-454b-a93f-04e5a65ab051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:51.329313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51679c48-4582-454b-a93f-04e5a65ab051 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:51.329601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51679c48-4582-454b-a93f-04e5a65ab051 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:51.329884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51679c48-4582-454b-a93f-04e5a65ab051 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:51.330787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51679c48-4582-454b-a93f-04e5a65ab051 None admin] RBAC: Adding query filter params (role.id=336f27e7f8624a30b30cef9acada97a0, scope.project.id=908a13d2b49544fbaf760b07e213bb7d, user.id=88cbd76f70d447c4ba92777b2c09821d) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:51.331099 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51679c48-4582-454b-a93f-04e5a65ab051 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:51.332532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51679c48-4582-454b-a93f-04e5a65ab051 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:51.338558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 344/678] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:41:51 2022] GET /identity/v3/role_assignments?user.id=88cbd76f70d447c4ba92777b2c09821d&scope.project.id=908a13d2b49544fbaf760b07e213bb7d&role.id=336f27e7f8624a30b30cef9acada97a0 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:51.339356 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51679c48-4582-454b-a93f-04e5a65ab051 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:51.343006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-b2a5a20f-ff08-4ecb-9fd6-9475ac898de2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:51.343006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:51.343006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:51.343006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:51.343006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:51.343006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:51.343006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:51.343006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:51.343006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:51.343006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:51.343006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:51.343006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:51.343006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:41:51.343006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:41:51.343006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:41:51.344003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:41:51.344003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:51.344003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:51.344003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:51.344003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:51.344003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:51.344003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:51.344003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:51.344003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:51.344003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:51.344003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:51.344003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:51.344003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:51.344003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:51.344003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:51.344003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:51.344003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:51.344890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:41:51.344890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:41:51.344890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:41:51.344890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:41:51.344890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:41:51.344890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:41:51.344890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:51.344890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:51.344890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 335/679] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:41:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:51.345991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5a20f-ff08-4ecb-9fd6-9475ac898de2 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:51.471238 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51679c48-4582-454b-a93f-04e5a65ab051 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:51.471456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5a20f-ff08-4ecb-9fd6-9475ac898de2 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:51.509617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66302eb9-12ad-4482-b653-d0e5078f5c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:51.510338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66302eb9-12ad-4482-b653-d0e5078f5c3a None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:51.510603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66302eb9-12ad-4482-b653-d0e5078f5c3a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:51.510874 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66302eb9-12ad-4482-b653-d0e5078f5c3a None admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:51.511674 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66302eb9-12ad-4482-b653-d0e5078f5c3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:51.511981 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66302eb9-12ad-4482-b653-d0e5078f5c3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:51.513452 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66302eb9-12ad-4482-b653-d0e5078f5c3a None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:51.513800 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245c6c8-efc6-42fb-86c9-0e8e339e4665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:51.514707 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245c6c8-efc6-42fb-86c9-0e8e339e4665 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:51.514995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245c6c8-efc6-42fb-86c9-0e8e339e4665 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:51.515403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245c6c8-efc6-42fb-86c9-0e8e339e4665 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:51.516259 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8245c6c8-efc6-42fb-86c9-0e8e339e4665 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:51.516580 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8245c6c8-efc6-42fb-86c9-0e8e339e4665 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:51.517987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8245c6c8-efc6-42fb-86c9-0e8e339e4665 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:51.521423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:51.533751 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 345/680] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:41:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:51.533751 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245c6c8-efc6-42fb-86c9-0e8e339e4665 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:51.534574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 336/681] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:41:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:51.535597 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66302eb9-12ad-4482-b653-d0e5078f5c3a None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:51.614331 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:51.665490 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245c6c8-efc6-42fb-86c9-0e8e339e4665 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:51.673028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66302eb9-12ad-4482-b653-d0e5078f5c3a None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:51.702940 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc899481-cc4d-4a38-970b-c953275406a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:51.703751 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc899481-cc4d-4a38-970b-c953275406a7 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:51.704034 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc899481-cc4d-4a38-970b-c953275406a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:51.704311 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc899481-cc4d-4a38-970b-c953275406a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:51.720179 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce038c7-bb5c-4863-82dd-9c536c4e8142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:51.720530 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc899481-cc4d-4a38-970b-c953275406a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:51.721457 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce038c7-bb5c-4863-82dd-9c536c4e8142 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:51.721980 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce038c7-bb5c-4863-82dd-9c536c4e8142 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:51.722148 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc899481-cc4d-4a38-970b-c953275406a7 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:51.722787 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce038c7-bb5c-4863-82dd-9c536c4e8142 None admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:51.724020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce038c7-bb5c-4863-82dd-9c536c4e8142 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:51.724488 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce038c7-bb5c-4863-82dd-9c536c4e8142 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:51.726443 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce038c7-bb5c-4863-82dd-9c536c4e8142 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:51.740320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-dc899481-cc4d-4a38-970b-c953275406a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:51.740320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:51.740320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:51.740320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:51.740320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:51.740320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:51.740320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:51.740320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:51.740320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:51.740320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:51.740320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:51.740320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:51.740320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:51.740320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:51.740320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:51.741454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:51.741454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:51.741454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:51.741454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:51.741454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:51.741454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:51.741454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:51.741454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:51.741454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:51.741454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:51.741454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:51.741454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:51.741454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:51.741454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:51.741454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:51.741454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:51.741454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:51.742404 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:51.742404 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:51.742404 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:51.742404 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:51.742404 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 346/682] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:51.742404 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc899481-cc4d-4a38-970b-c953275406a7 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:51.744423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 337/683] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:41:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:51.745647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce038c7-bb5c-4863-82dd-9c536c4e8142 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:51.815561 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:51.865482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce038c7-bb5c-4863-82dd-9c536c4e8142 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:51.866920 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc899481-cc4d-4a38-970b-c953275406a7 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:51.878280 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:51.907304 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645be6a6-640a-4d5b-8be6-c73ea9e4a7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:51.908205 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645be6a6-640a-4d5b-8be6-c73ea9e4a7d3 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:51.908752 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645be6a6-640a-4d5b-8be6-c73ea9e4a7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:51.909098 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645be6a6-640a-4d5b-8be6-c73ea9e4a7d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:51.912144 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6327816-9ddf-4981-a8a9-70fd38e64427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:51.912804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6327816-9ddf-4981-a8a9-70fd38e64427 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:51.913078 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6327816-9ddf-4981-a8a9-70fd38e64427 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:51.913361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6327816-9ddf-4981-a8a9-70fd38e64427 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:51.914241 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6327816-9ddf-4981-a8a9-70fd38e64427 None admin] RBAC: Adding query filter params (role.id=336f27e7f8624a30b30cef9acada97a0, scope.project.id=908a13d2b49544fbaf760b07e213bb7d, user.id=70ad51a8ad68440da6618891e2e08cb8) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:51.914551 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6327816-9ddf-4981-a8a9-70fd38e64427 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:51.915980 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6327816-9ddf-4981-a8a9-70fd38e64427 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:51.922405 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645be6a6-640a-4d5b-8be6-c73ea9e4a7d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:51.934046 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 338/684] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:41:51 2022] GET /identity/v3/role_assignments?user.id=70ad51a8ad68440da6618891e2e08cb8&scope.project.id=908a13d2b49544fbaf760b07e213bb7d&role.id=336f27e7f8624a30b30cef9acada97a0 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:51.934311 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645be6a6-640a-4d5b-8be6-c73ea9e4a7d3 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:51.934418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6327816-9ddf-4981-a8a9-70fd38e64427 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:51.955188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-645be6a6-640a-4d5b-8be6-c73ea9e4a7d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:51.955188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:51.955188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:51.955188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:51.955188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:51.955188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:51.955188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:51.955188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:51.955188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:51.955188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:51.955188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:51.955188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:51.955188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:51.955188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:51.955188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:51.956321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:51.956321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:51.956321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:51.956321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:51.956321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:51.956321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:51.956321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:51.956321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:51.956321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:51.956321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:51.956321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:51.956321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:51.956321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:51.956321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:51.956321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:51.956321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:51.956321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:51.957409 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:51.957409 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:51.957409 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:51.957409 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:51.957961 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 347/685] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:51.962044 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645be6a6-640a-4d5b-8be6-c73ea9e4a7d3 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:52.071757 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6327816-9ddf-4981-a8a9-70fd38e64427 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:52.107109 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:52.126302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2155b4-cd8f-4d7a-aa50-4664c7262165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:52.127237 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645be6a6-640a-4d5b-8be6-c73ea9e4a7d3 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:52.127396 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2155b4-cd8f-4d7a-aa50-4664c7262165 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:52.127893 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2155b4-cd8f-4d7a-aa50-4664c7262165 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:52.128474 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2155b4-cd8f-4d7a-aa50-4664c7262165 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:52.130212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2155b4-cd8f-4d7a-aa50-4664c7262165 None admin] RBAC: Adding query filter params (role.id=336f27e7f8624a30b30cef9acada97a0, scope.project.id=908a13d2b49544fbaf760b07e213bb7d, user.id=50dbd3dd7e87481cb940a467c294006c) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:52.131727 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2155b4-cd8f-4d7a-aa50-4664c7262165 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:52.133291 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2155b4-cd8f-4d7a-aa50-4664c7262165 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:52.146546 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 339/686] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:41:51 2022] GET /identity/v3/role_assignments?user.id=50dbd3dd7e87481cb940a467c294006c&scope.project.id=908a13d2b49544fbaf760b07e213bb7d&role.id=336f27e7f8624a30b30cef9acada97a0 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:52.148258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2155b4-cd8f-4d7a-aa50-4664c7262165 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:52.178903 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b5dae-def9-4079-a425-33d9681d849b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:52.180239 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b5dae-def9-4079-a425-33d9681d849b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:52.180699 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b5dae-def9-4079-a425-33d9681d849b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:52.181767 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b5dae-def9-4079-a425-33d9681d849b None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:52.183139 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b5dae-def9-4079-a425-33d9681d849b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:52.183653 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b5dae-def9-4079-a425-33d9681d849b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:52.186221 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b5dae-def9-4079-a425-33d9681d849b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:52.206131 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 348/687] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:52.211617 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b5dae-def9-4079-a425-33d9681d849b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:52.247984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:52.307623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2155b4-cd8f-4d7a-aa50-4664c7262165 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:52.362348 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b92501-12cd-401c-a584-e972761f1dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:52.362942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b92501-12cd-401c-a584-e972761f1dbe None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:52.363225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b92501-12cd-401c-a584-e972761f1dbe None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:52.363492 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b92501-12cd-401c-a584-e972761f1dbe None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:52.364233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b92501-12cd-401c-a584-e972761f1dbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:52.364536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b92501-12cd-401c-a584-e972761f1dbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:52.366366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b92501-12cd-401c-a584-e972761f1dbe None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:52.382755 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 340/688] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:52.387978 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b5dae-def9-4079-a425-33d9681d849b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:52.389059 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b92501-12cd-401c-a584-e972761f1dbe None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:52.441935 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe78605-b0a3-4eee-bf33-fe3a0d760707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:52.443162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe78605-b0a3-4eee-bf33-fe3a0d760707 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:52.443660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe78605-b0a3-4eee-bf33-fe3a0d760707 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:52.444147 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe78605-b0a3-4eee-bf33-fe3a0d760707 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:52.453843 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe78605-b0a3-4eee-bf33-fe3a0d760707 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:52.455678 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:52.456904 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe78605-b0a3-4eee-bf33-fe3a0d760707 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:52.476847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-9fe78605-b0a3-4eee-bf33-fe3a0d760707 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 14:41:52.476847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:52.476847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:52.476847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:52.476847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:52.476847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:52.476847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:52.476847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:52.476847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:52.476847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:52.476847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:52.476847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:52.476847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:41:52.476847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:41:52.476847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:41:52.476847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:41:52.477836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:52.477836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:52.477836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:41:52.477836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:52.477836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:41:52.477836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:52.477836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:52.477836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:52.477836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:52.477836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:52.477836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:52.477836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:52.477836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:52.477836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:52.477836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:52.477836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:52.477836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:52.477836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:52.478798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:52.478798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:52.478798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:41:52.478798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:41:52.478798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:41:52.478798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:41:52.478798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:41:52.478798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:41:52.478798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 14:41:52.478798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:52.485744 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 349/689] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:41:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:52.485888 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe78605-b0a3-4eee-bf33-fe3a0d760707 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:52.524858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b92501-12cd-401c-a584-e972761f1dbe None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:52.563810 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740933a6-7b3f-496c-bbe9-83686ee5190e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:52.564992 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740933a6-7b3f-496c-bbe9-83686ee5190e None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:52.565497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740933a6-7b3f-496c-bbe9-83686ee5190e None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:52.565972 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740933a6-7b3f-496c-bbe9-83686ee5190e None admin] PATH_INFO: `/v3/users/nova` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:52.574232 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740933a6-7b3f-496c-bbe9-83686ee5190e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:52.576006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740933a6-7b3f-496c-bbe9-83686ee5190e None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:52.580568 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:52.593431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-740933a6-7b3f-496c-bbe9-83686ee5190e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:41:52.593431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:52.593431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:52.593431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:52.593431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:52.593431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:52.593431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:52.593431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:52.593431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:52.593431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:52.593431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:52.593431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:52.593431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:41:52.593431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:41:52.593431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:41:52.593431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:41:52.594470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:52.594470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:52.594470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:41:52.594470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:52.594470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:41:52.594470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:52.594470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:52.594470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:52.594470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:52.594470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:52.594470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:52.594470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:52.594470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:52.594470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:52.594470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:52.594470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:52.594470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:52.594470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:52.595389 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:52.595389 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:52.595389 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:41:52.595389 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:41:52.595389 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:41:52.595389 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:41:52.595389 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:41:52.595389 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:41:52.595389 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:41:52.595389 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:52.598031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 341/690] 10.222.0.78 () {56 vars in 1093 bytes} [Wed May 25 14:41:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:52.603385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740933a6-7b3f-496c-bbe9-83686ee5190e None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:52.604645 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe78605-b0a3-4eee-bf33-fe3a0d760707 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:52.655373 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4a066-5f8a-4bff-bf2b-cd87acff5403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:52.656935 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4a066-5f8a-4bff-bf2b-cd87acff5403 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:52.657433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4a066-5f8a-4bff-bf2b-cd87acff5403 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:52.657932 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4a066-5f8a-4bff-bf2b-cd87acff5403 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:52.659034 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4a066-5f8a-4bff-bf2b-cd87acff5403 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:52.660027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4a066-5f8a-4bff-bf2b-cd87acff5403 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:52.662306 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4a066-5f8a-4bff-bf2b-cd87acff5403 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:52.688499 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 350/691] 10.222.0.78 () {56 vars in 1135 bytes} [Wed May 25 14:41:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:52.695213 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4a066-5f8a-4bff-bf2b-cd87acff5403 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:52.780546 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740933a6-7b3f-496c-bbe9-83686ee5190e None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:52.791602 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:52.829274 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4a066-5f8a-4bff-bf2b-cd87acff5403 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:52.898762 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b8217-91f7-45b0-a336-3aabf191606c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:52.899869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b8217-91f7-45b0-a336-3aabf191606c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:52.900362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b8217-91f7-45b0-a336-3aabf191606c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:52.900852 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b8217-91f7-45b0-a336-3aabf191606c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:52.912346 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c609db1-d7fc-474c-9580-dd63b6479590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:52.913493 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c609db1-d7fc-474c-9580-dd63b6479590 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:52.913982 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c609db1-d7fc-474c-9580-dd63b6479590 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:52.914492 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c609db1-d7fc-474c-9580-dd63b6479590 None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:52.916103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c609db1-d7fc-474c-9580-dd63b6479590 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:52.916651 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c609db1-d7fc-474c-9580-dd63b6479590 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:52.931813 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c609db1-d7fc-474c-9580-dd63b6479590 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:52.936000 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:52.993878 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 342/692] 10.222.0.78 () {56 vars in 1133 bytes} [Wed May 25 14:41:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:53.002484 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c609db1-d7fc-474c-9580-dd63b6479590 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:53.025350 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b8217-91f7-45b0-a336-3aabf191606c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:53.027167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b8217-91f7-45b0-a336-3aabf191606c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:53.159388 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 351/693] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:41:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:41:53.166683 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b8217-91f7-45b0-a336-3aabf191606c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:53.244018 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:53.266260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c609db1-d7fc-474c-9580-dd63b6479590 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:53.343486 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fe690-34be-4269-bf1c-ab35d3381369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:53.344522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fe690-34be-4269-bf1c-ab35d3381369 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:53.348482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fe690-34be-4269-bf1c-ab35d3381369 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:53.349045 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fe690-34be-4269-bf1c-ab35d3381369 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:53.372784 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b8217-91f7-45b0-a336-3aabf191606c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:53.399161 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:53.456376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128fe690-34be-4269-bf1c-ab35d3381369 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:53.458421 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128fe690-34be-4269-bf1c-ab35d3381369 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:53.471331 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b53cc-a305-4b1c-a103-7c72c516a45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:53.472513 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b53cc-a305-4b1c-a103-7c72c516a45d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:53.473000 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b53cc-a305-4b1c-a103-7c72c516a45d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:53.473521 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b53cc-a305-4b1c-a103-7c72c516a45d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:53.488009 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b53cc-a305-4b1c-a103-7c72c516a45d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:53.489896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b53cc-a305-4b1c-a103-7c72c516a45d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:53.509537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-b21b53cc-a305-4b1c-a103-7c72c516a45d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:53.509537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:53.509537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:53.509537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:53.509537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:53.509537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:53.509537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:53.509537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:53.509537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:53.509537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:53.509537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:53.509537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:53.509537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:53.509537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:53.509537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:53.510565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:53.510565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:53.510565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:53.510565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:53.510565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:53.510565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:53.510565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:53.510565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:53.510565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:53.510565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:53.510565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:53.510565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:53.510565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:53.510565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:53.510565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:53.510565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:53.510565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:53.511514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:53.511514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:53.511514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:53.511514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:53.514829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 352/694] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:53.520826 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b53cc-a305-4b1c-a103-7c72c516a45d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:53.642592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:53.655985 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b53cc-a305-4b1c-a103-7c72c516a45d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:53.695136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd8ba9-5307-42c5-9087-c233ee590c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:53.695827 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 343/695] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:41:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 694 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:41:53.696770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd8ba9-5307-42c5-9087-c233ee590c0c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:53.697063 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd8ba9-5307-42c5-9087-c233ee590c0c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:53.697320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd8ba9-5307-42c5-9087-c233ee590c0c None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:53.706051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd8ba9-5307-42c5-9087-c233ee590c0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:53.706051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd8ba9-5307-42c5-9087-c233ee590c0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:53.706051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd8ba9-5307-42c5-9087-c233ee590c0c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:53.708394 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fe690-34be-4269-bf1c-ab35d3381369 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:53.713704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 353/696] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:53.721480 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd8ba9-5307-42c5-9087-c233ee590c0c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:53.850601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd8ba9-5307-42c5-9087-c233ee590c0c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:53.874536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:53.932067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fe690-34be-4269-bf1c-ab35d3381369 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:53.946944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea5a5e-1971-421a-99cb-bb7da2c80ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:53.947767 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea5a5e-1971-421a-99cb-bb7da2c80ade None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:53.948056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea5a5e-1971-421a-99cb-bb7da2c80ade None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:53.948317 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea5a5e-1971-421a-99cb-bb7da2c80ade None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:53.964818 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ea5a5e-1971-421a-99cb-bb7da2c80ade None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:53.966633 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ea5a5e-1971-421a-99cb-bb7da2c80ade None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:53.993212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-68ea5a5e-1971-421a-99cb-bb7da2c80ade None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:53.993212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:53.993212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:53.993212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:53.993212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:53.993212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:53.993212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:53.993212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:53.993212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:53.993212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:53.993212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:53.993212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:53.993212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:41:53.993212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:41:53.993212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:41:53.994948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:41:53.994948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:53.994948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:53.994948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:53.994948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:53.994948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:53.994948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:53.994948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:53.994948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:53.994948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:53.994948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:53.994948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:53.994948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:53.994948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:53.994948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:53.994948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:53.994948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:53.996634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:41:53.996634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:41:53.996634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:41:53.996634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:41:53.996634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:41:53.996634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:41:53.996634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:53.996634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:53.996634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 354/697] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:41:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:41:54.010294 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865d4b6-c7ad-4afe-b6dd-fc3bbec20a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:54.011395 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea5a5e-1971-421a-99cb-bb7da2c80ade None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:54.011653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865d4b6-c7ad-4afe-b6dd-fc3bbec20a3a None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:54.012130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865d4b6-c7ad-4afe-b6dd-fc3bbec20a3a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:54.012620 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865d4b6-c7ad-4afe-b6dd-fc3bbec20a3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:54.024982 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865d4b6-c7ad-4afe-b6dd-fc3bbec20a3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:54.026924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865d4b6-c7ad-4afe-b6dd-fc3bbec20a3a None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:54.041032 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:54.054855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-b865d4b6-c7ad-4afe-b6dd-fc3bbec20a3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:54.054855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:54.054855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:54.054855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:54.054855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:54.054855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:54.054855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:54.054855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:54.054855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:54.054855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:54.054855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:54.054855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:54.054855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:54.054855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:54.054855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:54.055852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:54.055852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:54.055852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:54.055852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:54.055852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:54.055852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:54.055852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:54.055852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:54.055852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:54.055852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:54.055852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:54.055852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:54.055852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:54.055852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:54.055852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:54.055852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:54.055852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:54.056747 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:54.056747 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:54.056747 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:54.056747 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:54.061813 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 344/698] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:54.066904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865d4b6-c7ad-4afe-b6dd-fc3bbec20a3a None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:54.203159 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865d4b6-c7ad-4afe-b6dd-fc3bbec20a3a None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:54.222582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:54.242775 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327596d-a3c8-47cf-b976-2c0d4a78984c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:54.243829 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327596d-a3c8-47cf-b976-2c0d4a78984c None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:54.244498 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327596d-a3c8-47cf-b976-2c0d4a78984c None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:54.245364 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327596d-a3c8-47cf-b976-2c0d4a78984c None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:54.246674 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327596d-a3c8-47cf-b976-2c0d4a78984c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:54.247344 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327596d-a3c8-47cf-b976-2c0d4a78984c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:54.250213 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327596d-a3c8-47cf-b976-2c0d4a78984c None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:54.250956 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea5a5e-1971-421a-99cb-bb7da2c80ade None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:54.266397 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 345/699] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:54.269313 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1f4bf-265b-47ea-a9d7-f714ab16840f None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:54.269819 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1f4bf-265b-47ea-a9d7-f714ab16840f None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:54.270369 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1f4bf-265b-47ea-a9d7-f714ab16840f None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:54.272850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 346/700] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:54.273801 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1f4bf-265b-47ea-a9d7-f714ab16840f None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:54.290514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df79bed0-ea82-4abe-b86b-3fcf419227ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:54.291240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79bed0-ea82-4abe-b86b-3fcf419227ba None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:54.291527 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79bed0-ea82-4abe-b86b-3fcf419227ba None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:54.291804 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79bed0-ea82-4abe-b86b-3fcf419227ba None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:54.292595 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df79bed0-ea82-4abe-b86b-3fcf419227ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:54.292901 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df79bed0-ea82-4abe-b86b-3fcf419227ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:54.294374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df79bed0-ea82-4abe-b86b-3fcf419227ba None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:54.307726 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 355/701] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:41:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:54.309287 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56ad89-60c9-4dd2-88d4-4f8aa700858b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:54.309418 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56ad89-60c9-4dd2-88d4-4f8aa700858b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:54.309647 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56ad89-60c9-4dd2-88d4-4f8aa700858b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:54.328567 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-bb56ad89-60c9-4dd2-88d4-4f8aa700858b None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:54.330162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-bb56ad89-60c9-4dd2-88d4-4f8aa700858b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:54.400102 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1f4bf-265b-47ea-a9d7-f714ab16840f None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:54.415350 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:54.419125 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 356/702] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:54.421101 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56ad89-60c9-4dd2-88d4-4f8aa700858b None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:54.439422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451f624-d140-4e01-8723-12b10713cf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:54.440225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451f624-d140-4e01-8723-12b10713cf0d None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:54.440503 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451f624-d140-4e01-8723-12b10713cf0d None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:54.440773 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451f624-d140-4e01-8723-12b10713cf0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:54.448898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451f624-d140-4e01-8723-12b10713cf0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:54.450845 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451f624-d140-4e01-8723-12b10713cf0d None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:54.469303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-8451f624-d140-4e01-8723-12b10713cf0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:54.469303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:54.469303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:54.469303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:54.469303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:54.469303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:54.469303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:54.469303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:54.469303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:54.469303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:54.469303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:54.469303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:54.469303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:41:54.469303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:41:54.469303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:41:54.470330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:41:54.470330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:54.470330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:54.470330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:54.470330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:54.470330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:54.470330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:54.470330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:54.470330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:54.470330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:54.470330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:54.470330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:54.470330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:54.470330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:54.470330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:54.470330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:54.470330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:54.471260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:41:54.471260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:41:54.471260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:41:54.471260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:41:54.471260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:41:54.471260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:41:54.471260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:41:54.471260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:54.471260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 347/703] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:41:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:54.471260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451f624-d140-4e01-8723-12b10713cf0d None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:54.526539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:54.565602 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56ad89-60c9-4dd2-88d4-4f8aa700858b None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:54.615622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2fba96-5cb0-4639-94b4-28f413f75b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:54.616319 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fba96-5cb0-4639-94b4-28f413f75b3c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:54.616601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fba96-5cb0-4639-94b4-28f413f75b3c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:54.616876 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fba96-5cb0-4639-94b4-28f413f75b3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:54.617732 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2fba96-5cb0-4639-94b4-28f413f75b3c None admin] RBAC: Adding query filter params (role.id=336f27e7f8624a30b30cef9acada97a0, scope.project.id=908a13d2b49544fbaf760b07e213bb7d, user.id=fbe4a2d106d84fc9ac65c046ba675353) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:54.618070 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2fba96-5cb0-4639-94b4-28f413f75b3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:54.619963 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2fba96-5cb0-4639-94b4-28f413f75b3c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:54.626826 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 357/704] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:41:54 2022] GET /identity/v3/role_assignments?user.id=fbe4a2d106d84fc9ac65c046ba675353&scope.project.id=908a13d2b49544fbaf760b07e213bb7d&role.id=336f27e7f8624a30b30cef9acada97a0 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:54.627786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2fba96-5cb0-4639-94b4-28f413f75b3c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:54.628353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451f624-d140-4e01-8723-12b10713cf0d None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:54.694885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824915e-71a1-4b0a-9e74-c865268b34d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:54.696693 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824915e-71a1-4b0a-9e74-c865268b34d5 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:54.697375 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824915e-71a1-4b0a-9e74-c865268b34d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:54.698056 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824915e-71a1-4b0a-9e74-c865268b34d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:54.707716 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:54.714833 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b824915e-71a1-4b0a-9e74-c865268b34d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:54.716897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b824915e-71a1-4b0a-9e74-c865268b34d5 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:54.739802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-b824915e-71a1-4b0a-9e74-c865268b34d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:54.739802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:54.739802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:54.739802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:54.739802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:54.739802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:54.739802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:54.739802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:54.739802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:54.739802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:54.739802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:54.739802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:54.739802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:54.739802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:54.739802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:54.740811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:54.740811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:54.740811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:54.740811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:54.740811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:54.740811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:54.740811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:54.740811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:54.740811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:54.740811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:54.740811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:54.740811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:54.740811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:54.740811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:54.740811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:54.740811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:54.740811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:54.741745 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:54.741745 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:54.741745 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:54.741745 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:54.744396 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 348/705] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:54.750876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824915e-71a1-4b0a-9e74-c865268b34d5 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:54.769727 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2fba96-5cb0-4639-94b4-28f413f75b3c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:54.852992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb9c16-17bb-4699-8fc1-deed704ef8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:54.853593 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb9c16-17bb-4699-8fc1-deed704ef8d9 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:54.853875 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb9c16-17bb-4699-8fc1-deed704ef8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:54.854184 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb9c16-17bb-4699-8fc1-deed704ef8d9 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:54.854994 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb9c16-17bb-4699-8fc1-deed704ef8d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:54.855303 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb9c16-17bb-4699-8fc1-deed704ef8d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:54.856981 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb9c16-17bb-4699-8fc1-deed704ef8d9 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:54.888676 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 358/706] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:41:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:54.895929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb9c16-17bb-4699-8fc1-deed704ef8d9 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:54.961658 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:54.991136 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824915e-71a1-4b0a-9e74-c865268b34d5 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:55.036861 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69577d3f-b7f1-4e29-835c-636fbac770e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:55.037976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69577d3f-b7f1-4e29-835c-636fbac770e3 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:55.038533 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69577d3f-b7f1-4e29-835c-636fbac770e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:55.039005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69577d3f-b7f1-4e29-835c-636fbac770e3 None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:55.040018 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69577d3f-b7f1-4e29-835c-636fbac770e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:55.040527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69577d3f-b7f1-4e29-835c-636fbac770e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:55.042338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69577d3f-b7f1-4e29-835c-636fbac770e3 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:55.056750 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 349/707] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:55.057393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb9c16-17bb-4699-8fc1-deed704ef8d9 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:55.062687 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69577d3f-b7f1-4e29-835c-636fbac770e3 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:55.087270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:55.101202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fadddad-6d91-41f8-b7fd-7be5d1b606a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:55.101737 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fadddad-6d91-41f8-b7fd-7be5d1b606a8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:55.102042 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fadddad-6d91-41f8-b7fd-7be5d1b606a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:55.102509 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fadddad-6d91-41f8-b7fd-7be5d1b606a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:55.103752 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fadddad-6d91-41f8-b7fd-7be5d1b606a8 None admin] RBAC: Adding query filter params (role.id=e566b37cd92f467198b5dc40f183effc, scope.project.id=908a13d2b49544fbaf760b07e213bb7d, user.id=e34e1d7ffbac49b0bde45829af0c6925) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:55.104019 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fadddad-6d91-41f8-b7fd-7be5d1b606a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:55.105475 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fadddad-6d91-41f8-b7fd-7be5d1b606a8 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:55.112859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 359/708] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:41:54 2022] GET /identity/v3/role_assignments?user.id=e34e1d7ffbac49b0bde45829af0c6925&scope.project.id=908a13d2b49544fbaf760b07e213bb7d&role.id=e566b37cd92f467198b5dc40f183effc => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:55.189047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c8b6e-56e4-4e00-b63a-7f0cdc9e1228 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:55.189390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c8b6e-56e4-4e00-b63a-7f0cdc9e1228 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:55.189617 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c8b6e-56e4-4e00-b63a-7f0cdc9e1228 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:55.191457 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 360/709] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:41:55.191616 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69577d3f-b7f1-4e29-835c-636fbac770e3 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:55.197802 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f316745-eac7-4303-b332-784427ed5002 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:55.198148 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f316745-eac7-4303-b332-784427ed5002 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:55.198424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f316745-eac7-4303-b332-784427ed5002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:55.222331 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-4f316745-eac7-4303-b332-784427ed5002 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:55.224566 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4f316745-eac7-4303-b332-784427ed5002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:55.233353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bc840-cbf9-4241-88e2-631aeabf5bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:55.234344 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bc840-cbf9-4241-88e2-631aeabf5bf7 None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:55.234742 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bc840-cbf9-4241-88e2-631aeabf5bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:55.235021 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bc840-cbf9-4241-88e2-631aeabf5bf7 None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:55.235739 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50bc840-cbf9-4241-88e2-631aeabf5bf7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:55.237426 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50bc840-cbf9-4241-88e2-631aeabf5bf7 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:55.268689 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 350/710] 10.222.0.78 () {60 vars in 1132 bytes} [Wed May 25 14:41:55 2022] POST /identity/v3/users => generated 254 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:41:55.270404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5826aaa-0875-458a-9f69-27103fd48b37 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:55.270738 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5826aaa-0875-458a-9f69-27103fd48b37 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:55.271020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5826aaa-0875-458a-9f69-27103fd48b37 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:55.273024 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 351/711] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:55.281614 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf04dea-69e8-4b39-81be-dc7b41cf9573 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:55.282083 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf04dea-69e8-4b39-81be-dc7b41cf9573 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:55.282502 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf04dea-69e8-4b39-81be-dc7b41cf9573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:55.292539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:55.320764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-daf04dea-69e8-4b39-81be-dc7b41cf9573 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:55.323461 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-daf04dea-69e8-4b39-81be-dc7b41cf9573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:55.364473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 361/712] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:55.388987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f316745-eac7-4303-b332-784427ed5002 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:55.464449 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 352/713] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:41:55.509751 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf04dea-69e8-4b39-81be-dc7b41cf9573 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:55.533814 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f316745-eac7-4303-b332-784427ed5002 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:55.554543 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:55.575978 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1396fc-11cc-49b2-91cd-ecd48659cf8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:55.577860 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1396fc-11cc-49b2-91cd-ecd48659cf8c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:55.578391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1396fc-11cc-49b2-91cd-ecd48659cf8c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:55.579011 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1396fc-11cc-49b2-91cd-ecd48659cf8c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:55.580001 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1396fc-11cc-49b2-91cd-ecd48659cf8c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:55.581639 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1396fc-11cc-49b2-91cd-ecd48659cf8c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:55.598057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-6c1396fc-11cc-49b2-91cd-ecd48659cf8c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:41:55.598057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:55.598057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:41:55.598057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:41:55.598057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:41:55.598057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:41:55.598057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:41:55.598057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:41:55.598057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:41:55.598057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:55.598057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:41:55.598057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:55.598057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:55.598057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:55.598057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:55.598057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:55.599190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:55.599190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:55.599190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:55.599190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:55.599190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:55.599190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:55.599190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:55.599190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:41:55.599190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:41:55.599190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:41:55.599190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:41:55.599190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:55.599190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:55.599190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:55.599190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:55.599190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:55.600047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:55.600047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:55.600047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:55.600047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:55.600047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:55.600047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:55.600047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:55.600047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:55.600047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:55.600047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:55.600047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:55.600047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:41:55.600047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:41:55.600047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:41:55.600047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:55.606136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 362/714] 10.222.0.78 () {56 vars in 1129 bytes} [Wed May 25 14:41:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:55.610371 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1396fc-11cc-49b2-91cd-ecd48659cf8c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:55.650152 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf04dea-69e8-4b39-81be-dc7b41cf9573 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:55.693339 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cce73-9574-4aa1-88bd-4e2f97a6416f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:55.695186 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cce73-9574-4aa1-88bd-4e2f97a6416f None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:55.695746 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cce73-9574-4aa1-88bd-4e2f97a6416f None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:55.696224 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cce73-9574-4aa1-88bd-4e2f97a6416f None admin] PATH_INFO: `/v3/services/network` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:55.697114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1cce73-9574-4aa1-88bd-4e2f97a6416f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:55.698774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1cce73-9574-4aa1-88bd-4e2f97a6416f None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:55.701561 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:55.720113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-1b1cce73-9574-4aa1-88bd-4e2f97a6416f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:41:55.720113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:55.720113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:41:55.720113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:41:55.720113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:41:55.720113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:41:55.720113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:41:55.720113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:41:55.720113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:41:55.720113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:55.720113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:41:55.720113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:55.720113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:55.720113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:55.720113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:55.720113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:55.720113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:55.722253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:55.722253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:55.722253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:55.722253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:55.722253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:55.722253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:55.722253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:41:55.722253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:41:55.722253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:41:55.722253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:41:55.722253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:55.722253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:55.722253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:55.722253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:55.722253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:55.722253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:55.725058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:55.725058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:55.725058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:55.725058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:55.725058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:55.725058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:55.725058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:55.725058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:55.725058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:55.725058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:55.725058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:41:55.725058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:41:55.725058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:41:55.725058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:55.725058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 353/715] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:41:55 2022] GET /identity/v3/services/network => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:41:55.728656 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cce73-9574-4aa1-88bd-4e2f97a6416f None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:55.771255 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1396fc-11cc-49b2-91cd-ecd48659cf8c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:55.853633 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cce73-9574-4aa1-88bd-4e2f97a6416f None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:55.905890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd912fc-4e79-4c37-b62f-50f86fff4cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:55.906623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd912fc-4e79-4c37-b62f-50f86fff4cc9 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:55.906891 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd912fc-4e79-4c37-b62f-50f86fff4cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:55.907148 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd912fc-4e79-4c37-b62f-50f86fff4cc9 None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:55.907826 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd912fc-4e79-4c37-b62f-50f86fff4cc9 None admin] RBAC: Adding query filter params (name=network) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:55.908129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd912fc-4e79-4c37-b62f-50f86fff4cc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:55.909543 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd912fc-4e79-4c37-b62f-50f86fff4cc9 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:55.917189 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a82b6-0d37-45ce-859c-0fac321d5860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:55.918789 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a82b6-0d37-45ce-859c-0fac321d5860 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:55.919580 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a82b6-0d37-45ce-859c-0fac321d5860 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:55.920933 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a82b6-0d37-45ce-859c-0fac321d5860 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:55.922630 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a82b6-0d37-45ce-859c-0fac321d5860 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:55.923463 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a82b6-0d37-45ce-859c-0fac321d5860 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:55.923993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 354/716] 10.222.0.78 () {56 vars in 1112 bytes} [Wed May 25 14:41:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:55.927192 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a82b6-0d37-45ce-859c-0fac321d5860 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:55.929262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd912fc-4e79-4c37-b62f-50f86fff4cc9 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:55.937173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 363/717] 10.222.0.78 () {56 vars in 1124 bytes} [Wed May 25 14:41:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:55.945658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a82b6-0d37-45ce-859c-0fac321d5860 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:56.018638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:56.077907 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:56.132028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a82b6-0d37-45ce-859c-0fac321d5860 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:56.133268 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd912fc-4e79-4c37-b62f-50f86fff4cc9 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:56.178787 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff2afc4-1a38-4d44-a95f-1a42a0453b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:56.179375 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2afc4-1a38-4d44-a95f-1a42a0453b8a None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:56.179653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2afc4-1a38-4d44-a95f-1a42a0453b8a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:56.179923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2afc4-1a38-4d44-a95f-1a42a0453b8a None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:56.180645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff2afc4-1a38-4d44-a95f-1a42a0453b8a None admin] RBAC: Adding query filter params (type=network) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:56.180946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff2afc4-1a38-4d44-a95f-1a42a0453b8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:56.182354 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff2afc4-1a38-4d44-a95f-1a42a0453b8a None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:56.203354 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 355/718] 10.222.0.78 () {56 vars in 1112 bytes} [Wed May 25 14:41:55 2022] GET /identity/v3/services?type=network => generated 124 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:56.211902 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343f068-1dcb-4050-bd83-62d5644232f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:56.212597 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343f068-1dcb-4050-bd83-62d5644232f8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:56.212883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343f068-1dcb-4050-bd83-62d5644232f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:56.213160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343f068-1dcb-4050-bd83-62d5644232f8 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:56.213881 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a343f068-1dcb-4050-bd83-62d5644232f8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:56.214214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a343f068-1dcb-4050-bd83-62d5644232f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:56.215607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a343f068-1dcb-4050-bd83-62d5644232f8 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:56.227683 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 364/719] 10.222.0.78 () {56 vars in 1124 bytes} [Wed May 25 14:41:55 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:57.856788 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bbd81a-81a0-4bc4-9b8f-c4e4ef73f609 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:57.857379 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bbd81a-81a0-4bc4-9b8f-c4e4ef73f609 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:57.857379 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bbd81a-81a0-4bc4-9b8f-c4e4ef73f609 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:57.859018 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 356/720] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:57.864768 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05732340-b105-4636-91ea-23e4a4b8e759 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:57.865290 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05732340-b105-4636-91ea-23e4a4b8e759 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:57.865368 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05732340-b105-4636-91ea-23e4a4b8e759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:57.884374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-05732340-b105-4636-91ea-23e4a4b8e759 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:57.885593 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-05732340-b105-4636-91ea-23e4a4b8e759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:58.032719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 365/721] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:58.076368 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bbd81a-81a0-4bc4-9b8f-c4e4ef73f609 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:58.255531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a7eea-009f-4b43-8e35-d34175e28b8c None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:58.255819 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a7eea-009f-4b43-8e35-d34175e28b8c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:58.256218 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a7eea-009f-4b43-8e35-d34175e28b8c None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:58.260853 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 366/722] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:58.265195 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec507a-98aa-40bb-9f0b-20de048ebc69 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:58.265714 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec507a-98aa-40bb-9f0b-20de048ebc69 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:58.266161 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec507a-98aa-40bb-9f0b-20de048ebc69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:58.297590 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-1aec507a-98aa-40bb-9f0b-20de048ebc69 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:58.298826 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1aec507a-98aa-40bb-9f0b-20de048ebc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:58.321413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bbd81a-81a0-4bc4-9b8f-c4e4ef73f609 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:58.408778 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:58.413538 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:58.438945 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392dec8-3d69-48fb-86f0-ee107ea38281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:58.440233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392dec8-3d69-48fb-86f0-ee107ea38281 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:58.440683 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392dec8-3d69-48fb-86f0-ee107ea38281 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:58.441137 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392dec8-3d69-48fb-86f0-ee107ea38281 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:58.459651 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392dec8-3d69-48fb-86f0-ee107ea38281 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:58.475168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-b392dec8-3d69-48fb-86f0-ee107ea38281 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:41:58.475168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:58.475168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:58.475168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:58.475168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:58.475168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:58.475168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:58.475168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:58.475168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:58.475168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:58.475168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:58.475168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:58.475168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:58.475168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:58.475168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:58.475168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:58.476163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:58.476163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:58.476163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:58.476163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:58.476163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:58.476163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:58.476163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:58.476163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:58.476163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:58.476163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:58.476163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:58.476163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:58.476163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:58.476163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:58.476163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:58.476163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:58.477022 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:58.477022 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:58.477022 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:58.477022 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:58.477022 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:58.477022 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:58.477022 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:58.477022 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:58.477022 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:58.477022 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:58.477022 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:41:58.477022 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:58.477658 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 357/723] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:41:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:41:58.483107 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4477f-fb30-4efe-996c-d2bdfb49713f None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:58.483107 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4477f-fb30-4efe-996c-d2bdfb49713f None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:58.483107 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4477f-fb30-4efe-996c-d2bdfb49713f None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:58.483751 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 367/724] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:58.483870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 358/725] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:58.488676 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec507a-98aa-40bb-9f0b-20de048ebc69 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:58.491832 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461d8a8-dab7-4e5e-a703-81492b7a7b8a None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:58.492277 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461d8a8-dab7-4e5e-a703-81492b7a7b8a None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:58.492728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461d8a8-dab7-4e5e-a703-81492b7a7b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:58.521962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-2461d8a8-dab7-4e5e-a703-81492b7a7b8a None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:58.523838 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-2461d8a8-dab7-4e5e-a703-81492b7a7b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:58.679869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec507a-98aa-40bb-9f0b-20de048ebc69 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:58.771064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0441c8-d685-420e-b69e-8889463d71c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:58.772282 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0441c8-d685-420e-b69e-8889463d71c2 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:58.772708 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0441c8-d685-420e-b69e-8889463d71c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:58.773194 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0441c8-d685-420e-b69e-8889463d71c2 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:58.774609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0441c8-d685-420e-b69e-8889463d71c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:58.775153 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0441c8-d685-420e-b69e-8889463d71c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:58.777940 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0441c8-d685-420e-b69e-8889463d71c2 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:58.802387 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 368/726] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:41:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:58.804082 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0441c8-d685-420e-b69e-8889463d71c2 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:58.810136 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 359/727] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:58.821889 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461d8a8-dab7-4e5e-a703-81492b7a7b8a None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:58.877258 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:59.005813 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0441c8-d685-420e-b69e-8889463d71c2 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:59.012937 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461d8a8-dab7-4e5e-a703-81492b7a7b8a None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:59.042849 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:59.068821 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0967d14-5076-4e5a-bf7c-af67a64b42d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:59.070134 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0967d14-5076-4e5a-bf7c-af67a64b42d2 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:59.070603 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0967d14-5076-4e5a-bf7c-af67a64b42d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:59.071065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0967d14-5076-4e5a-bf7c-af67a64b42d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:59.072188 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad42df-7821-4d96-a95c-25c10110e6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:59.073412 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad42df-7821-4d96-a95c-25c10110e6dd None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:59.073818 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad42df-7821-4d96-a95c-25c10110e6dd None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:59.074290 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad42df-7821-4d96-a95c-25c10110e6dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:59.088745 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0967d14-5076-4e5a-bf7c-af67a64b42d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:59.090299 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad42df-7821-4d96-a95c-25c10110e6dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:59.092348 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad42df-7821-4d96-a95c-25c10110e6dd None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:59.096953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-b0967d14-5076-4e5a-bf7c-af67a64b42d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:41:59.096953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:59.096953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:59.096953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:59.096953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:59.096953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:59.096953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:59.096953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:59.096953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:59.096953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:59.096953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:59.096953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:59.096953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:59.096953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:59.096953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:59.096953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:59.097870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:59.097870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:59.097870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:59.097870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:59.097870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:59.097870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:59.097870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:59.097870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:59.097870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:59.097870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:59.097870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:59.097870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:59.097870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:59.097870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:59.097870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:59.097870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:59.099015 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:59.099015 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:59.099015 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:59.099015 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:59.099015 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:59.099015 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:59.099015 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:59.099015 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:59.099015 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:59.099015 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:59.099015 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:41:59.099015 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:59.099015 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 369/728] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:41:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:59.099737 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0967d14-5076-4e5a-bf7c-af67a64b42d2 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:59.112434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-fbad42df-7821-4d96-a95c-25c10110e6dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:59.112434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:59.112434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:59.112434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:59.112434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:59.112434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:59.112434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:59.112434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:59.112434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:59.112434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:59.112434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:59.112434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:59.112434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:59.112434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:59.112434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:59.113347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:59.113347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:59.113347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:59.113347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:59.113347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:59.113347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:59.113347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:59.113347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:59.113347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:59.113347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:59.113347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:59.113347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:59.113347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:59.113347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:59.113347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:59.113347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:59.113347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:59.114315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:59.114315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:59.114315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:59.114315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:59.114315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 360/729] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:41:59.115298 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156c6ea-154d-4b1b-9935-666fcc58a8c5 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:59.115298 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156c6ea-154d-4b1b-9935-666fcc58a8c5 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:59.115460 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156c6ea-154d-4b1b-9935-666fcc58a8c5 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:59.118140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 361/730] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:59.118140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156c6ea-154d-4b1b-9935-666fcc58a8c5 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:59.209250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:59.220086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0967d14-5076-4e5a-bf7c-af67a64b42d2 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:59.236866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156c6ea-154d-4b1b-9935-666fcc58a8c5 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:59.256376 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5cee7-b56c-4874-8e22-1aa36d10c75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:59.257166 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5cee7-b56c-4874-8e22-1aa36d10c75d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:59.257450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5cee7-b56c-4874-8e22-1aa36d10c75d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:59.257722 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5cee7-b56c-4874-8e22-1aa36d10c75d None admin] PATH_INFO: `/v3/roles/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:59.268520 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f5cee7-b56c-4874-8e22-1aa36d10c75d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:41:59.276241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-76f5cee7-b56c-4874-8e22-1aa36d10c75d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:59.276241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:59.276241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:59.276241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:59.276241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:59.276241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:59.276241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:59.276241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:59.276241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:59.276241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:59.276241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:59.276241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:59.276241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:41:59.276241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:41:59.276241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:41:59.276241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:41:59.277233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:41:59.277233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:41:59.277233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:41:59.277233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:41:59.277233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:59.277233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:59.277233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:59.277233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:59.277233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:59.277233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:59.277233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:59.277233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:59.277233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:59.277233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:59.277233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:59.277233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:59.278128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:59.278128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:59.278128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:59.278128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:59.278128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:41:59.278128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:41:59.278128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:41:59.278128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:41:59.278128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:41:59.278128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:41:59.278128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:41:59.278128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:59.278128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 370/731] 10.222.0.78 () {56 vars in 1102 bytes} [Wed May 25 14:41:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:59.278970 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5cee7-b56c-4874-8e22-1aa36d10c75d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:59.280489 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce04f6-8afc-4b32-8ed7-3242ecd202d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:59.281166 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce04f6-8afc-4b32-8ed7-3242ecd202d8 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:59.281911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce04f6-8afc-4b32-8ed7-3242ecd202d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:59.282216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce04f6-8afc-4b32-8ed7-3242ecd202d8 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:59.283335 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ce04f6-8afc-4b32-8ed7-3242ecd202d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:59.283655 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ce04f6-8afc-4b32-8ed7-3242ecd202d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:59.285048 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ce04f6-8afc-4b32-8ed7-3242ecd202d8 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:59.298484 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 362/732] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:41:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:59.300327 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd4d3a-b148-438a-bac3-cd620d5bdd90 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:59.300856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd4d3a-b148-438a-bac3-cd620d5bdd90 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:59.301519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd4d3a-b148-438a-bac3-cd620d5bdd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:59.319611 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-e6cd4d3a-b148-438a-bac3-cd620d5bdd90 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:59.321018 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-e6cd4d3a-b148-438a-bac3-cd620d5bdd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:59.350552 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:59.391899 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5cee7-b56c-4874-8e22-1aa36d10c75d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:59.396931 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 363/733] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:59.397770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd4d3a-b148-438a-bac3-cd620d5bdd90 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:59.428079 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094c4fc-59ba-4b0f-959c-64b1d724d001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:59.428748 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094c4fc-59ba-4b0f-959c-64b1d724d001 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:59.429037 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094c4fc-59ba-4b0f-959c-64b1d724d001 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:59.429321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094c4fc-59ba-4b0f-959c-64b1d724d001 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:59.430073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094c4fc-59ba-4b0f-959c-64b1d724d001 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:59.430382 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094c4fc-59ba-4b0f-959c-64b1d724d001 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:59.431788 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094c4fc-59ba-4b0f-959c-64b1d724d001 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:59.444412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 371/734] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:59.445291 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094c4fc-59ba-4b0f-959c-64b1d724d001 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:59.463128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:59.499595 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd4d3a-b148-438a-bac3-cd620d5bdd90 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:59.533787 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9755230-845e-4d3f-aa97-4307c2b69a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:59.534476 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9755230-845e-4d3f-aa97-4307c2b69a78 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:59.534748 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9755230-845e-4d3f-aa97-4307c2b69a78 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:59.535014 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9755230-845e-4d3f-aa97-4307c2b69a78 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:59.536008 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9755230-845e-4d3f-aa97-4307c2b69a78 None admin] RBAC: Adding query filter params (name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:59.536307 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9755230-845e-4d3f-aa97-4307c2b69a78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:59.537674 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9755230-845e-4d3f-aa97-4307c2b69a78 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:59.546287 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 364/735] 10.222.0.78 () {56 vars in 1103 bytes} [Wed May 25 14:41:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:59.547531 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215068a-f519-4682-8104-4b9e90614138 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:59.547817 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215068a-f519-4682-8104-4b9e90614138 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:59.548088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215068a-f519-4682-8104-4b9e90614138 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:59.549524 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 365/736] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:41:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:41:59.550333 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094c4fc-59ba-4b0f-959c-64b1d724d001 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:59.554068 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215068a-f519-4682-8104-4b9e90614138 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:59.577478 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:59.585771 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc801d92-7d4b-4d91-a11b-c0fd81e86ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:59.586595 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc801d92-7d4b-4d91-a11b-c0fd81e86ca8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:59.586885 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc801d92-7d4b-4d91-a11b-c0fd81e86ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:59.587170 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc801d92-7d4b-4d91-a11b-c0fd81e86ca8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:59.598928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc801d92-7d4b-4d91-a11b-c0fd81e86ca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:59.600399 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc801d92-7d4b-4d91-a11b-c0fd81e86ca8 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:59.619774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-dc801d92-7d4b-4d91-a11b-c0fd81e86ca8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:59.619774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:59.619774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:59.619774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:59.619774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:59.619774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:59.619774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:59.619774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:59.619774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:59.619774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:59.619774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:59.619774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:59.619774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:59.619774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:59.619774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:59.620870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:59.620870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:59.620870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:59.620870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:59.620870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:59.620870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:59.620870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:59.620870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:59.620870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:59.620870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:59.620870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:59.620870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:59.620870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:59.620870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:59.620870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:59.620870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:59.620870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:59.621863 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:59.621863 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:59.621863 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:59.621863 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:59.626151 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 372/737] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:59.626964 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc801d92-7d4b-4d91-a11b-c0fd81e86ca8 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:59.658643 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215068a-f519-4682-8104-4b9e90614138 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:59.687409 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:59.692783 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140c56d-b320-476a-9803-502ff28b7cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:59.693437 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140c56d-b320-476a-9803-502ff28b7cfa None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:59.693714 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140c56d-b320-476a-9803-502ff28b7cfa None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:59.693984 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140c56d-b320-476a-9803-502ff28b7cfa None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:59.694732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140c56d-b320-476a-9803-502ff28b7cfa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:59.696092 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140c56d-b320-476a-9803-502ff28b7cfa None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:59.710889 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 366/738] 10.222.0.78 () {60 vars in 1141 bytes} [Wed May 25 14:41:59 2022] POST /identity/v3/services => generated 251 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:59.711694 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140c56d-b320-476a-9803-502ff28b7cfa None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:59.738305 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc801d92-7d4b-4d91-a11b-c0fd81e86ca8 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:59.788881 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d0dd4-6b4f-47e3-a402-4ef62a23fa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:59.789723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d0dd4-6b4f-47e3-a402-4ef62a23fa9f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:59.789982 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d0dd4-6b4f-47e3-a402-4ef62a23fa9f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:59.790283 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d0dd4-6b4f-47e3-a402-4ef62a23fa9f None admin] PATH_INFO: `/v3/users/swift` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:59.799201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d0dd4-6b4f-47e3-a402-4ef62a23fa9f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:59.800749 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d0dd4-6b4f-47e3-a402-4ef62a23fa9f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:59.809750 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:59.816693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-8f1d0dd4-6b4f-47e3-a402-4ef62a23fa9f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 14:41:59.816693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:59.816693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:59.816693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:59.816693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:59.816693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:59.816693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:59.816693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:59.816693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:59.816693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:59.816693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:59.816693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:59.816693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:41:59.816693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:41:59.816693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:41:59.816693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:41:59.817736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:59.817736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:59.817736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:41:59.817736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:59.817736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:41:59.817736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:41:59.817736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:59.817736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:59.817736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:59.817736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:59.817736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:59.817736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:41:59.817736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:59.817736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:59.817736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:59.817736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:59.817736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:59.817736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:59.819095 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:59.819095 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:59.819095 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:41:59.819095 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:41:59.819095 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:41:59.819095 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:41:59.819095 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:41:59.819095 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:41:59.819095 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 14:41:59.819095 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:41:59.819095 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 373/739] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:41:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:59.819694 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e074e-e506-4114-a575-b03a4652e70e None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:59.819694 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e074e-e506-4114-a575-b03a4652e70e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:59.820143 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e074e-e506-4114-a575-b03a4652e70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:59.839247 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-211e074e-e506-4114-a575-b03a4652e70e None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:59.840495 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-211e074e-e506-4114-a575-b03a4652e70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:59.841402 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140c56d-b320-476a-9803-502ff28b7cfa None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:59.879444 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2f644-6fb3-41bc-8ac3-2227ebbe4398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:59.880252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2f644-6fb3-41bc-8ac3-2227ebbe4398 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:59.880665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2f644-6fb3-41bc-8ac3-2227ebbe4398 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:59.880839 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2f644-6fb3-41bc-8ac3-2227ebbe4398 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:59.894989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc2f644-6fb3-41bc-8ac3-2227ebbe4398 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:59.895623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc2f644-6fb3-41bc-8ac3-2227ebbe4398 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:59.914246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-2fc2f644-6fb3-41bc-8ac3-2227ebbe4398 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:59.914246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:41:59.914246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:41:59.914246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:41:59.914246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:41:59.914246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:41:59.914246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:41:59.914246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:41:59.914246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:41:59.914246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:41:59.914246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:41:59.914246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:41:59.914246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:41:59.914246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:41:59.914246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:41:59.915154 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:41:59.915154 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:41:59.915154 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:41:59.915154 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:41:59.915154 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:41:59.915154 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:41:59.915154 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:41:59.915154 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:41:59.915154 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:41:59.915154 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:41:59.915154 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:41:59.915154 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:41:59.915154 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:41:59.915154 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:41:59.915154 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:41:59.915154 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:41:59.915154 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:41:59.916328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:41:59.916328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:41:59.916328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:41:59.916328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:41:59.916328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 367/740] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:41:59.916328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2f644-6fb3-41bc-8ac3-2227ebbe4398 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:59.930777 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 374/741] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:41:59 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:41:59.932395 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e074e-e506-4114-a575-b03a4652e70e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:59.992006 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:00.053998 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e074e-e506-4114-a575-b03a4652e70e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:00.067136 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2f644-6fb3-41bc-8ac3-2227ebbe4398 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:00.107000 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a9125-cc9a-4c28-9760-1495c12e0673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:00.108540 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a9125-cc9a-4c28-9760-1495c12e0673 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:00.109056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a9125-cc9a-4c28-9760-1495c12e0673 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:00.109552 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a9125-cc9a-4c28-9760-1495c12e0673 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:00.111348 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a9125-cc9a-4c28-9760-1495c12e0673 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:00.112085 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a9125-cc9a-4c28-9760-1495c12e0673 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:00.114620 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a9125-cc9a-4c28-9760-1495c12e0673 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:00.118697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898e25b-b248-4916-87f7-be40829e451c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:00.119385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898e25b-b248-4916-87f7-be40829e451c None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:00.119672 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898e25b-b248-4916-87f7-be40829e451c None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:00.119948 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898e25b-b248-4916-87f7-be40829e451c None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:00.120762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898e25b-b248-4916-87f7-be40829e451c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:00.121071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898e25b-b248-4916-87f7-be40829e451c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:00.122761 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898e25b-b248-4916-87f7-be40829e451c None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:00.125748 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:00.140284 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 368/742] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:00.141152 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898e25b-b248-4916-87f7-be40829e451c None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:00.152628 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 375/743] 10.222.0.78 () {56 vars in 1135 bytes} [Wed May 25 14:41:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:00.153749 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a9125-cc9a-4c28-9760-1495c12e0673 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:00.259468 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898e25b-b248-4916-87f7-be40829e451c None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:00.270059 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a9125-cc9a-4c28-9760-1495c12e0673 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:00.280900 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:00.298399 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3f102-7b06-4932-a70d-7245df6321dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:00.299097 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3f102-7b06-4932-a70d-7245df6321dc None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:00.299324 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3f102-7b06-4932-a70d-7245df6321dc None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:00.299593 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3f102-7b06-4932-a70d-7245df6321dc None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:00.300313 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3f102-7b06-4932-a70d-7245df6321dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:00.300613 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3f102-7b06-4932-a70d-7245df6321dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:00.301985 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3f102-7b06-4932-a70d-7245df6321dc None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:00.309226 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e2239-d081-4530-b5c2-90686cafe541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:00.309841 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e2239-d081-4530-b5c2-90686cafe541 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:00.310157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e2239-d081-4530-b5c2-90686cafe541 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:00.310980 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e2239-d081-4530-b5c2-90686cafe541 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:00.311711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e2239-d081-4530-b5c2-90686cafe541 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:00.313097 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e2239-d081-4530-b5c2-90686cafe541 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:00.316381 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 369/744] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:00.318235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3f102-7b06-4932-a70d-7245df6321dc None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:00.330673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 376/745] 10.222.0.78 () {60 vars in 1141 bytes} [Wed May 25 14:42:00 2022] POST /identity/v3/services => generated 240 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:42:00.331805 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e2239-d081-4530-b5c2-90686cafe541 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:00.426491 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:00.474200 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e2239-d081-4530-b5c2-90686cafe541 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:00.483834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3f102-7b06-4932-a70d-7245df6321dc None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:00.523340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16de8c-2655-4530-98fc-8bbd31bc6009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:00.524155 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16de8c-2655-4530-98fc-8bbd31bc6009 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:00.524433 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16de8c-2655-4530-98fc-8bbd31bc6009 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:00.524704 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16de8c-2655-4530-98fc-8bbd31bc6009 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:00.528764 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d249f1-9807-4aa9-b5bb-e53c035fc85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:00.529863 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d249f1-9807-4aa9-b5bb-e53c035fc85a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:00.530390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d249f1-9807-4aa9-b5bb-e53c035fc85a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:00.530863 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d249f1-9807-4aa9-b5bb-e53c035fc85a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:00.533915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b16de8c-2655-4530-98fc-8bbd31bc6009 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:00.535411 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b16de8c-2655-4530-98fc-8bbd31bc6009 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:00.552598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-1b16de8c-2655-4530-98fc-8bbd31bc6009 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:42:00.552598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:00.552598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:00.552598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:00.552598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:00.552598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:00.552598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:00.552598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:00.552598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:00.552598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:00.552598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:00.552598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:00.552598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:42:00.552598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:42:00.552598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:42:00.552598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:42:00.553966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:00.553966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:00.553966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:42:00.553966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:00.553966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:42:00.553966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:00.553966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:00.553966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:00.553966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:00.553966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:00.553966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:00.553966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:00.553966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:00.553966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:00.553966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:00.553966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:00.553966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:00.553966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:00.554946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:00.554946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:00.554946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:42:00.554946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:42:00.554946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:42:00.554946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:42:00.554946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:42:00.554946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:42:00.554946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:42:00.554946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:00.554946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 370/746] 10.222.0.78 () {56 vars in 1093 bytes} [Wed May 25 14:42:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:00.554946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16de8c-2655-4530-98fc-8bbd31bc6009 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:00.581294 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:00.643390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d249f1-9807-4aa9-b5bb-e53c035fc85a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:00.645829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d249f1-9807-4aa9-b5bb-e53c035fc85a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:00.698002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16de8c-2655-4530-98fc-8bbd31bc6009 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:00.719396 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:00.748879 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6d4fb-803a-429e-a86f-87941d8606bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:00.750296 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6d4fb-803a-429e-a86f-87941d8606bf None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:00.750730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6d4fb-803a-429e-a86f-87941d8606bf None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:00.751415 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6d4fb-803a-429e-a86f-87941d8606bf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:00.762831 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6d4fb-803a-429e-a86f-87941d8606bf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:00.763352 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 377/747] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:42:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:42:00.764766 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d249f1-9807-4aa9-b5bb-e53c035fc85a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:00.765729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6d4fb-803a-429e-a86f-87941d8606bf None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:00.785995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-1ff6d4fb-803a-429e-a86f-87941d8606bf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:42:00.785995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:00.785995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:00.785995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:00.785995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:00.785995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:00.785995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:00.785995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:00.785995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:00.785995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:00.785995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:00.785995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:00.785995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:42:00.785995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:42:00.785995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:42:00.785995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:42:00.787237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:00.787237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:00.787237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:42:00.787237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:00.787237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:42:00.787237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:00.787237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:00.787237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:00.787237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:00.787237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:00.787237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:00.787237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:00.787237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:00.787237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:00.787237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:00.787237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:00.787237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:00.787237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:00.788399 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:00.788399 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:00.788399 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:42:00.788399 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:42:00.788399 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:42:00.788399 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:42:00.788399 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:42:00.788399 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:42:00.788399 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:42:00.788399 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:00.795312 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 371/748] 10.222.0.78 () {56 vars in 1117 bytes} [Wed May 25 14:42:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:00.795312 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6d4fb-803a-429e-a86f-87941d8606bf None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:00.861939 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:00.915388 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6d4fb-803a-429e-a86f-87941d8606bf None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:00.921366 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d249f1-9807-4aa9-b5bb-e53c035fc85a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:00.958490 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7dc32-5b44-4b8d-832c-a6db6c0ba727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:00.959427 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7dc32-5b44-4b8d-832c-a6db6c0ba727 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:00.959664 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7dc32-5b44-4b8d-832c-a6db6c0ba727 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:00.959940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7dc32-5b44-4b8d-832c-a6db6c0ba727 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:00.969515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afc75c-adc1-4d23-a574-067cb8540e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:00.971430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afc75c-adc1-4d23-a574-067cb8540e1a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:00.972173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afc75c-adc1-4d23-a574-067cb8540e1a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:00.972606 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afc75c-adc1-4d23-a574-067cb8540e1a None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:00.972764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7dc32-5b44-4b8d-832c-a6db6c0ba727 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:00.973796 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7afc75c-adc1-4d23-a574-067cb8540e1a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:00.974310 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7dc32-5b44-4b8d-832c-a6db6c0ba727 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:00.974426 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7afc75c-adc1-4d23-a574-067cb8540e1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:00.976027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7afc75c-adc1-4d23-a574-067cb8540e1a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:00.995574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-29b7dc32-5b44-4b8d-832c-a6db6c0ba727 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:00.995574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:00.995574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:00.995574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:00.995574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:00.995574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:00.995574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:00.995574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:00.995574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:00.995574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:00.995574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:00.995574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:00.995574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:00.995574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:00.995574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:00.996673 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:00.996673 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:00.996673 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:00.996673 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:00.996673 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:00.996673 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:00.996673 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:00.996673 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:00.996673 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:00.996673 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:00.996673 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:00.996673 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:00.996673 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:00.996673 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:00.996673 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:00.996673 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:00.996673 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:00.997549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:00.997549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:00.997549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:00.997549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:00.997549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 372/749] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:00.997549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7dc32-5b44-4b8d-832c-a6db6c0ba727 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:01.007359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 378/750] 10.222.0.78 () {56 vars in 1133 bytes} [Wed May 25 14:42:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:01.007359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afc75c-adc1-4d23-a574-067cb8540e1a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:01.038242 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:01.126362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afc75c-adc1-4d23-a574-067cb8540e1a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:01.127307 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7dc32-5b44-4b8d-832c-a6db6c0ba727 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:01.156751 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:01.171754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ad47c-64e8-4079-a0e1-d02388ebca57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:01.172746 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ad47c-64e8-4079-a0e1-d02388ebca57 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:01.173175 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ad47c-64e8-4079-a0e1-d02388ebca57 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:01.173640 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ad47c-64e8-4079-a0e1-d02388ebca57 None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:01.174109 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dff1ce-ef72-40ce-bce6-9e5c9ff0cf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:01.175037 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dff1ce-ef72-40ce-bce6-9e5c9ff0cf7b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:01.175351 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ad47c-64e8-4079-a0e1-d02388ebca57 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:01.175456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dff1ce-ef72-40ce-bce6-9e5c9ff0cf7b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:01.175746 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dff1ce-ef72-40ce-bce6-9e5c9ff0cf7b None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:01.176121 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ad47c-64e8-4079-a0e1-d02388ebca57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:01.176667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dff1ce-ef72-40ce-bce6-9e5c9ff0cf7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:01.177138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dff1ce-ef72-40ce-bce6-9e5c9ff0cf7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:01.178334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ad47c-64e8-4079-a0e1-d02388ebca57 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:01.178903 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dff1ce-ef72-40ce-bce6-9e5c9ff0cf7b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:01.199438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 379/751] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:01.201211 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dff1ce-ef72-40ce-bce6-9e5c9ff0cf7b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:01.207319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 373/752] 10.222.0.78 () {56 vars in 1149 bytes} [Wed May 25 14:42:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:01.209695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ad47c-64e8-4079-a0e1-d02388ebca57 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:01.306417 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:01.320513 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dff1ce-ef72-40ce-bce6-9e5c9ff0cf7b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:01.324964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ad47c-64e8-4079-a0e1-d02388ebca57 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:01.359624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d4d35-f69a-4c6c-b725-1cbe020f4d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:01.360859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d4d35-f69a-4c6c-b725-1cbe020f4d2e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:01.361607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d4d35-f69a-4c6c-b725-1cbe020f4d2e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:01.362253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d4d35-f69a-4c6c-b725-1cbe020f4d2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:01.363262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d2c28-3a5d-448b-9454-5e2254cbfcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:01.364155 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d2c28-3a5d-448b-9454-5e2254cbfcb9 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:01.364443 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d2c28-3a5d-448b-9454-5e2254cbfcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:01.364717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d2c28-3a5d-448b-9454-5e2254cbfcb9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:01.372232 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d2c28-3a5d-448b-9454-5e2254cbfcb9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:01.373751 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d2c28-3a5d-448b-9454-5e2254cbfcb9 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:01.376314 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d4d35-f69a-4c6c-b725-1cbe020f4d2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:01.378084 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d4d35-f69a-4c6c-b725-1cbe020f4d2e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:01.389162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-8d6d2c28-3a5d-448b-9454-5e2254cbfcb9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:01.389162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:01.389162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:01.389162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:01.389162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:01.389162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:01.389162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:01.389162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:01.389162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:01.389162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:01.389162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:01.389162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:01.389162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:42:01.389162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:42:01.389162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:42:01.390269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:42:01.390269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:01.390269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:01.390269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:01.390269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:01.390269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:01.390269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:01.390269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:01.390269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:01.390269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:01.390269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:01.390269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:01.390269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:01.390269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:01.390269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:01.390269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:01.390269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:01.391275 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:42:01.391275 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:42:01.391275 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:42:01.391275 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:42:01.391275 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:42:01.391275 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:42:01.391275 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:01.391275 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:01.391275 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 374/753] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:42:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:01.391763 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d2c28-3a5d-448b-9454-5e2254cbfcb9 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:01.401162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-a55d4d35-f69a-4c6c-b725-1cbe020f4d2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:01.401162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:01.401162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:01.401162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:01.401162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:01.401162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:01.401162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:01.401162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:01.401162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:01.401162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:01.401162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:01.401162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:01.401162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:01.401162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:01.401162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:01.402118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:01.402118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:01.402118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:01.402118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:01.402118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:01.402118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:01.402118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:01.402118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:01.402118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:01.402118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:01.402118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:01.402118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:01.402118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:01.402118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:01.402118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:01.402118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:01.402118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:01.403007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:01.403007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:01.403007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:01.403007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:01.410096 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 380/754] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:01.411569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d4d35-f69a-4c6c-b725-1cbe020f4d2e None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:01.450695 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:01.504678 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d2c28-3a5d-448b-9454-5e2254cbfcb9 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:01.527120 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d4d35-f69a-4c6c-b725-1cbe020f4d2e None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:01.543985 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2daeaf-d243-46fb-ba4b-a6086a410d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:01.545179 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2daeaf-d243-46fb-ba4b-a6086a410d94 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:01.545484 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2daeaf-d243-46fb-ba4b-a6086a410d94 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:01.545765 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2daeaf-d243-46fb-ba4b-a6086a410d94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:01.562614 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:01.569002 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c3126-2a15-4076-98c1-6a57aaa6e7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:01.569969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c3126-2a15-4076-98c1-6a57aaa6e7d2 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:01.570485 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c3126-2a15-4076-98c1-6a57aaa6e7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:01.570957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c3126-2a15-4076-98c1-6a57aaa6e7d2 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:01.572061 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c3126-2a15-4076-98c1-6a57aaa6e7d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:01.572567 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c3126-2a15-4076-98c1-6a57aaa6e7d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:01.574348 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c3126-2a15-4076-98c1-6a57aaa6e7d2 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:01.589345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 381/755] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:42:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:01.590549 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c3126-2a15-4076-98c1-6a57aaa6e7d2 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:01.623772 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2daeaf-d243-46fb-ba4b-a6086a410d94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:01.625287 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2daeaf-d243-46fb-ba4b-a6086a410d94 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:01.699549 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:01.709013 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c3126-2a15-4076-98c1-6a57aaa6e7d2 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:01.736442 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 375/756] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:42:01 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:42:01.736442 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2daeaf-d243-46fb-ba4b-a6086a410d94 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:01.760169 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc01e6-646d-4043-8c82-f3d2ac1009a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:01.760852 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc01e6-646d-4043-8c82-f3d2ac1009a4 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:01.761108 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc01e6-646d-4043-8c82-f3d2ac1009a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:01.761355 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc01e6-646d-4043-8c82-f3d2ac1009a4 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:01.762111 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc01e6-646d-4043-8c82-f3d2ac1009a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:01.762423 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc01e6-646d-4043-8c82-f3d2ac1009a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:01.764439 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc01e6-646d-4043-8c82-f3d2ac1009a4 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:01.782234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 382/757] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:01.783332 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc01e6-646d-4043-8c82-f3d2ac1009a4 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:01.791340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:01.859540 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2daeaf-d243-46fb-ba4b-a6086a410d94 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:01.900004 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636b33d-044e-4fdd-a41b-8b51225d74a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:01.901039 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636b33d-044e-4fdd-a41b-8b51225d74a9 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:01.901550 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636b33d-044e-4fdd-a41b-8b51225d74a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:01.902076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636b33d-044e-4fdd-a41b-8b51225d74a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:01.903140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636b33d-044e-4fdd-a41b-8b51225d74a9 None admin] RBAC: Adding query filter params (role.id=e566b37cd92f467198b5dc40f183effc, scope.project.id=908a13d2b49544fbaf760b07e213bb7d, user.id=fbe4a2d106d84fc9ac65c046ba675353) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:01.903667 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636b33d-044e-4fdd-a41b-8b51225d74a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:01.904735 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc01e6-646d-4043-8c82-f3d2ac1009a4 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:01.906358 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636b33d-044e-4fdd-a41b-8b51225d74a9 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:01.913627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 376/758] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:42:01 2022] GET /identity/v3/role_assignments?user.id=fbe4a2d106d84fc9ac65c046ba675353&scope.project.id=908a13d2b49544fbaf760b07e213bb7d&role.id=e566b37cd92f467198b5dc40f183effc => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:01.914894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636b33d-044e-4fdd-a41b-8b51225d74a9 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:01.954144 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0bf98-55e9-4fc6-8177-80202d61cbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:01.954857 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0bf98-55e9-4fc6-8177-80202d61cbaa None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:01.955139 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0bf98-55e9-4fc6-8177-80202d61cbaa None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:01.955412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0bf98-55e9-4fc6-8177-80202d61cbaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:01.968581 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0bf98-55e9-4fc6-8177-80202d61cbaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:01.970185 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0bf98-55e9-4fc6-8177-80202d61cbaa None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:01.990300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-eab0bf98-55e9-4fc6-8177-80202d61cbaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:01.990300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:01.990300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:01.990300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:01.990300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:01.990300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:01.990300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:01.990300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:01.990300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:01.990300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:01.990300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:01.990300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:01.990300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:01.990300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:01.990300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:01.991275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:01.991275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:01.991275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:01.991275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:01.991275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:01.991275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:01.991275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:01.991275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:01.991275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:01.991275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:01.991275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:01.991275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:01.991275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:01.991275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:01.991275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:01.991275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:01.991275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:01.992155 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:01.992155 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:01.992155 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:01.992155 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:01.992155 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 383/759] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:01.997105 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0bf98-55e9-4fc6-8177-80202d61cbaa None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:02.008958 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:02.040105 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636b33d-044e-4fdd-a41b-8b51225d74a9 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:02.051438 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:02.081460 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf6050-70c9-428d-bff7-b53d4ffcc365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:02.082732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf6050-70c9-428d-bff7-b53d4ffcc365 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:02.083259 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf6050-70c9-428d-bff7-b53d4ffcc365 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:02.083769 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf6050-70c9-428d-bff7-b53d4ffcc365 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:02.092377 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf6050-70c9-428d-bff7-b53d4ffcc365 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:02.094363 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf6050-70c9-428d-bff7-b53d4ffcc365 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:02.109931 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-4abf6050-70c9-428d-bff7-b53d4ffcc365 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:02.109931 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:02.109931 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:02.109931 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:02.109931 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:02.109931 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:02.109931 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:02.109931 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:02.109931 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:02.109931 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:02.109931 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:02.109931 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:02.109931 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:42:02.109931 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:42:02.109931 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:42:02.111071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:42:02.111071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:02.111071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:02.111071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:02.111071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:02.111071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:02.111071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:02.111071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:02.111071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:02.111071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:02.111071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:02.111071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:02.111071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:02.111071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:02.111071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:02.111071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:02.111071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:02.111964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:42:02.111964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:42:02.111964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:42:02.111964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:42:02.111964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:42:02.111964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:42:02.111964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:02.111964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:02.114664 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 377/760] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:42:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:02.115864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0bf98-55e9-4fc6-8177-80202d61cbaa None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:02.116582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4f6d9-23ad-477a-8060-6164a104786e None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:02.117079 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4f6d9-23ad-477a-8060-6164a104786e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:02.117577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4f6d9-23ad-477a-8060-6164a104786e None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:02.119709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 378/761] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:02.127372 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4f6d9-23ad-477a-8060-6164a104786e None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:02.174300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4191488-ce8f-42a1-8e52-152275001d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:02.175394 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4191488-ce8f-42a1-8e52-152275001d66 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:02.175919 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4191488-ce8f-42a1-8e52-152275001d66 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:02.176398 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4191488-ce8f-42a1-8e52-152275001d66 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:02.177355 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4191488-ce8f-42a1-8e52-152275001d66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:02.177781 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4191488-ce8f-42a1-8e52-152275001d66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:02.179362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4191488-ce8f-42a1-8e52-152275001d66 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:02.195763 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 384/762] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:02.203132 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114dcb4a-bb9b-464c-9cb4-fd3ca35dcec4 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:02.203132 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114dcb4a-bb9b-464c-9cb4-fd3ca35dcec4 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:02.203132 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114dcb4a-bb9b-464c-9cb4-fd3ca35dcec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:02.225441 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-114dcb4a-bb9b-464c-9cb4-fd3ca35dcec4 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:02.227254 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-114dcb4a-bb9b-464c-9cb4-fd3ca35dcec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:02.292469 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4f6d9-23ad-477a-8060-6164a104786e None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:02.312469 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:02.330337 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 385/763] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:02.331389 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114dcb4a-bb9b-464c-9cb4-fd3ca35dcec4 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:02.352593 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6c0ce-1bc3-47d1-ae47-e374a8d5f5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:02.353231 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6c0ce-1bc3-47d1-ae47-e374a8d5f5d8 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:02.353615 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6c0ce-1bc3-47d1-ae47-e374a8d5f5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:02.353991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6c0ce-1bc3-47d1-ae47-e374a8d5f5d8 None admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:02.354936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6c0ce-1bc3-47d1-ae47-e374a8d5f5d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:02.355354 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6c0ce-1bc3-47d1-ae47-e374a8d5f5d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:02.356886 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6c0ce-1bc3-47d1-ae47-e374a8d5f5d8 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:02.370866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 379/764] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:42:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:02.376979 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6c0ce-1bc3-47d1-ae47-e374a8d5f5d8 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:02.406803 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:02.487419 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114dcb4a-bb9b-464c-9cb4-fd3ca35dcec4 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:02.504327 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6c0ce-1bc3-47d1-ae47-e374a8d5f5d8 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:02.542799 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ddf99-701a-4104-b83b-e89b1ead9269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:02.544153 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ddf99-701a-4104-b83b-e89b1ead9269 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:02.544632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ddf99-701a-4104-b83b-e89b1ead9269 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:02.545891 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ddf99-701a-4104-b83b-e89b1ead9269 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:02.552250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b6d9d-ee06-46c5-94c1-ae028374316d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:02.553230 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b6d9d-ee06-46c5-94c1-ae028374316d None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:02.553629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b6d9d-ee06-46c5-94c1-ae028374316d None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:02.554001 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b6d9d-ee06-46c5-94c1-ae028374316d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:02.554794 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b6d9d-ee06-46c5-94c1-ae028374316d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:02.555675 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ddf99-701a-4104-b83b-e89b1ead9269 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:02.556458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b6d9d-ee06-46c5-94c1-ae028374316d None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:02.557440 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ddf99-701a-4104-b83b-e89b1ead9269 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:02.573361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-b52ddf99-701a-4104-b83b-e89b1ead9269 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:02.573361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:02.573361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:02.573361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:02.573361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:02.573361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:02.573361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:02.573361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:02.573361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:02.573361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:02.573361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:02.573361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:02.573361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:42:02.573361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:42:02.573361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:42:02.574557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:42:02.574557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:02.574557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:02.574557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:02.574557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:02.574557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:02.574557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:02.574557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:02.574557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:02.574557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:02.574557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:02.574557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:02.574557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:02.574557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:02.574557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:02.574557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:02.574557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:02.575578 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:42:02.575578 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:42:02.575578 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:42:02.575578 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:42:02.575578 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:42:02.575578 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:42:02.575578 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:02.575578 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:02.577306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-b28b6d9d-ee06-46c5-94c1-ae028374316d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:42:02.577306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:02.577306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:42:02.577306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:42:02.577306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:42:02.577306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:42:02.577306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:42:02.577306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:02.577306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:42:02.577306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:02.577306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:42:02.577306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:02.577306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:02.577306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:02.577306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:02.577306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:02.578509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:02.578509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:02.578509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:02.578509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:02.578509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:02.578509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:02.578509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:02.578509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:42:02.578509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:42:02.578509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:42:02.578509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:42:02.578509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:02.578509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:02.578509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:02.578509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:02.578509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:02.579547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:02.579547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:02.579547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:02.579547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:02.579547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:02.579547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:02.579547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:02.579547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:02.579547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:02.579547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:02.579547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:02.579547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:42:02.579547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:02.579547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:42:02.579547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:02.585378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 380/765] 10.222.0.78 () {56 vars in 1129 bytes} [Wed May 25 14:42:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:02.585882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 386/766] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:42:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:02.587128 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b6d9d-ee06-46c5-94c1-ae028374316d None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:02.587322 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ddf99-701a-4104-b83b-e89b1ead9269 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:02.676282 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:02.707443 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:02.720952 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b6d9d-ee06-46c5-94c1-ae028374316d None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:02.736871 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ddf99-701a-4104-b83b-e89b1ead9269 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:02.767472 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9763c-d78b-4109-9b9a-0b486e9f03ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:02.768703 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9763c-d78b-4109-9b9a-0b486e9f03ff None admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:02.769078 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9763c-d78b-4109-9b9a-0b486e9f03ff None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:02.769451 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9763c-d78b-4109-9b9a-0b486e9f03ff None admin] PATH_INFO: `/v3/projects/908a13d2b49544fbaf760b07e213bb7d/users/e34e1d7ffbac49b0bde45829af0c6925/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:02.772220 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe9763c-d78b-4109-9b9a-0b486e9f03ff None admin] RBAC: Authorizing `identity:create_grant(project_id=908a13d2b49544fbaf760b07e213bb7d, user_id=e34e1d7ffbac49b0bde45829af0c6925, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:02.773691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe9763c-d78b-4109-9b9a-0b486e9f03ff None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:02.779878 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbb7dd-0b62-49db-b13a-fbdf3098e3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:02.781080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbb7dd-0b62-49db-b13a-fbdf3098e3b3 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:02.781608 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbb7dd-0b62-49db-b13a-fbdf3098e3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:02.782190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbb7dd-0b62-49db-b13a-fbdf3098e3b3 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:02.783268 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cbb7dd-0b62-49db-b13a-fbdf3098e3b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:02.783799 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cbb7dd-0b62-49db-b13a-fbdf3098e3b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:02.785910 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cbb7dd-0b62-49db-b13a-fbdf3098e3b3 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:02.798038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 381/767] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:42:02 2022] PUT /identity/v3/projects/908a13d2b49544fbaf760b07e213bb7d/users/e34e1d7ffbac49b0bde45829af0c6925/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:02.798038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9763c-d78b-4109-9b9a-0b486e9f03ff None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:02.807448 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 387/768] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:42:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:02.815294 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbb7dd-0b62-49db-b13a-fbdf3098e3b3 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:02.920837 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9763c-d78b-4109-9b9a-0b486e9f03ff None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:02.960472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbb7dd-0b62-49db-b13a-fbdf3098e3b3 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:02.962105 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db3629-25ca-40d1-ba5b-3c5692487626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:02.962850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db3629-25ca-40d1-ba5b-3c5692487626 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:02.963069 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db3629-25ca-40d1-ba5b-3c5692487626 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:02.963346 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db3629-25ca-40d1-ba5b-3c5692487626 None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:02.964081 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db3629-25ca-40d1-ba5b-3c5692487626 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:02.964386 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db3629-25ca-40d1-ba5b-3c5692487626 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:02.965820 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db3629-25ca-40d1-ba5b-3c5692487626 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:02.971701 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 382/769] 10.222.0.78 () {56 vars in 1124 bytes} [Wed May 25 14:42:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:02.978607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8705fbc-9263-42c5-946a-419c403e7fbc None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:02.978607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8705fbc-9263-42c5-946a-419c403e7fbc None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:02.978607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8705fbc-9263-42c5-946a-419c403e7fbc None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:02.981410 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 383/770] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:02.981410 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8705fbc-9263-42c5-946a-419c403e7fbc None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:03.002313 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:03.002530 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:03.009004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184718a1-b345-4e36-8813-06dead5f7009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:03.010341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184718a1-b345-4e36-8813-06dead5f7009 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:03.010341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184718a1-b345-4e36-8813-06dead5f7009 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:03.010577 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184718a1-b345-4e36-8813-06dead5f7009 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:03.011515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184718a1-b345-4e36-8813-06dead5f7009 None admin] RBAC: Adding query filter params (role.id=336f27e7f8624a30b30cef9acada97a0, scope.project.id=908a13d2b49544fbaf760b07e213bb7d, user.id=b2935579c8554bed8b1e211c47548ac5) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:03.012097 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184718a1-b345-4e36-8813-06dead5f7009 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:03.015026 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184718a1-b345-4e36-8813-06dead5f7009 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:03.021925 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 388/771] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:42:02 2022] GET /identity/v3/role_assignments?user.id=b2935579c8554bed8b1e211c47548ac5&scope.project.id=908a13d2b49544fbaf760b07e213bb7d&role.id=336f27e7f8624a30b30cef9acada97a0 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:03.027699 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023541c-9c7f-4420-8cec-58cd127b8142 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:03.031214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023541c-9c7f-4420-8cec-58cd127b8142 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:03.031579 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023541c-9c7f-4420-8cec-58cd127b8142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:03.071292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-5023541c-9c7f-4420-8cec-58cd127b8142 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:03.072586 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5023541c-9c7f-4420-8cec-58cd127b8142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:03.114322 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8705fbc-9263-42c5-946a-419c403e7fbc None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:03.159682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a01004-0467-4196-868d-39c2481b4aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:03.161374 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a01004-0467-4196-868d-39c2481b4aff None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:03.161822 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a01004-0467-4196-868d-39c2481b4aff None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:03.162273 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a01004-0467-4196-868d-39c2481b4aff None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:03.163246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a01004-0467-4196-868d-39c2481b4aff None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:03.163695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a01004-0467-4196-868d-39c2481b4aff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:03.163882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 389/772] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:03.165463 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a01004-0467-4196-868d-39c2481b4aff None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:03.172360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 384/773] 10.222.0.78 () {56 vars in 1124 bytes} [Wed May 25 14:42:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:03.178412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023541c-9c7f-4420-8cec-58cd127b8142 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:03.218320 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a01004-0467-4196-868d-39c2481b4aff None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:03.327143 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:03.349546 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a01004-0467-4196-868d-39c2481b4aff None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:03.350339 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023541c-9c7f-4420-8cec-58cd127b8142 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:03.362777 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:03.390046 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ce00e-8ee5-4b6f-84c2-1b827fa480c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:03.390777 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3db2c-91bf-41f8-a30a-6574f5232d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:03.391453 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3db2c-91bf-41f8-a30a-6574f5232d9e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:03.391741 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3db2c-91bf-41f8-a30a-6574f5232d9e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:03.391974 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ce00e-8ee5-4b6f-84c2-1b827fa480c2 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:03.392075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3db2c-91bf-41f8-a30a-6574f5232d9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:03.392528 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ce00e-8ee5-4b6f-84c2-1b827fa480c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:03.392815 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b3db2c-91bf-41f8-a30a-6574f5232d9e None admin] RBAC: Adding query filter params (interface=public, service_id=e81e23f1e8c543d39aa9d296b4fd9c9e, region_id=RegionOne) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:03.393129 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b3db2c-91bf-41f8-a30a-6574f5232d9e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:03.393414 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ce00e-8ee5-4b6f-84c2-1b827fa480c2 None admin] PATH_INFO: `/v3/services/network` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:03.394313 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595ce00e-8ee5-4b6f-84c2-1b827fa480c2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:03.394550 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b3db2c-91bf-41f8-a30a-6574f5232d9e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:03.396092 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595ce00e-8ee5-4b6f-84c2-1b827fa480c2 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:03.403756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 390/774] 10.222.0.78 () {56 vars in 1251 bytes} [Wed May 25 14:42:03 2022] GET /identity/v3/endpoints?service_id=e81e23f1e8c543d39aa9d296b4fd9c9e&interface=public®ion_id=RegionOne => generated 194 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:03.418050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3db2c-91bf-41f8-a30a-6574f5232d9e None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:03.418256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-595ce00e-8ee5-4b6f-84c2-1b827fa480c2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:42:03.418256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:03.418256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:42:03.418256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:42:03.418256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:42:03.418256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:42:03.418256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:42:03.418256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:03.418256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:42:03.418256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:03.418256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:42:03.418256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:03.418256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:03.418256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:03.418256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:03.418256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:03.418256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:03.419356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:03.419356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:03.419356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:03.419356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:03.419356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:03.419356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:03.419356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:42:03.419356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:42:03.419356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:42:03.419356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:42:03.419356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:03.419356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:03.419356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:03.419356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:03.419356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:03.419356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:03.420235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:03.420235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:03.420235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:03.420235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:03.420235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:03.420235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:03.420235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:03.420235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:03.420235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:03.420235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:03.420235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:42:03.420235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:03.420235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:42:03.420235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:03.420235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 385/775] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:42:03 2022] GET /identity/v3/services/network => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:03.421035 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ce00e-8ee5-4b6f-84c2-1b827fa480c2 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:03.525287 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3db2c-91bf-41f8-a30a-6574f5232d9e None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:03.563196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1d478-0f9c-4322-87e3-6760c1ad273e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:03.563907 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1d478-0f9c-4322-87e3-6760c1ad273e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:03.564192 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1d478-0f9c-4322-87e3-6760c1ad273e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:03.564469 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1d478-0f9c-4322-87e3-6760c1ad273e None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:03.565391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1d478-0f9c-4322-87e3-6760c1ad273e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:03.566902 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1d478-0f9c-4322-87e3-6760c1ad273e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:03.576670 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 391/776] 10.222.0.78 () {56 vars in 1087 bytes} [Wed May 25 14:42:03 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:03.669477 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ce00e-8ee5-4b6f-84c2-1b827fa480c2 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:03.725227 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718287e4-5a69-4971-bf36-56413141bc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:03.725921 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718287e4-5a69-4971-bf36-56413141bc98 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:03.726231 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718287e4-5a69-4971-bf36-56413141bc98 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:03.726510 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718287e4-5a69-4971-bf36-56413141bc98 None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:03.727256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718287e4-5a69-4971-bf36-56413141bc98 None admin] RBAC: Adding query filter params (name=network) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:03.727568 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718287e4-5a69-4971-bf36-56413141bc98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:03.729005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718287e4-5a69-4971-bf36-56413141bc98 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:03.737348 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 386/777] 10.222.0.78 () {56 vars in 1112 bytes} [Wed May 25 14:42:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:03.754769 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1d478-0f9c-4322-87e3-6760c1ad273e None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:03.801598 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:03.892306 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1d478-0f9c-4322-87e3-6760c1ad273e None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:03.930565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241a862-9f34-40b1-9d1d-6c048c090abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:03.931170 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241a862-9f34-40b1-9d1d-6c048c090abb None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:03.931454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241a862-9f34-40b1-9d1d-6c048c090abb None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:03.931731 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241a862-9f34-40b1-9d1d-6c048c090abb None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:03.932452 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1241a862-9f34-40b1-9d1d-6c048c090abb None admin] RBAC: Adding query filter params (type=network) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:03.932762 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1241a862-9f34-40b1-9d1d-6c048c090abb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:03.934670 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1241a862-9f34-40b1-9d1d-6c048c090abb None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:03.942315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 392/778] 10.222.0.78 () {56 vars in 1112 bytes} [Wed May 25 14:42:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:03.947805 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718287e4-5a69-4971-bf36-56413141bc98 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:03.974405 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:04.104125 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718287e4-5a69-4971-bf36-56413141bc98 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:04.151827 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f2660-9147-49f3-bf38-9d792657b4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:04.152552 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f2660-9147-49f3-bf38-9d792657b4b8 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:04.152840 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f2660-9147-49f3-bf38-9d792657b4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:04.153113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f2660-9147-49f3-bf38-9d792657b4b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:04.154461 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f2660-9147-49f3-bf38-9d792657b4b8 None admin] RBAC: Adding query filter params (interface=public, service_id=4e6e945867694fb1820774c866c702bd, region_id=RegionOne) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:04.154778 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f2660-9147-49f3-bf38-9d792657b4b8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:04.156162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f2660-9147-49f3-bf38-9d792657b4b8 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:04.173609 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 387/779] 10.222.0.78 () {56 vars in 1251 bytes} [Wed May 25 14:42:03 2022] GET /identity/v3/endpoints?service_id=4e6e945867694fb1820774c866c702bd&interface=public®ion_id=RegionOne => generated 194 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:04.178934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241a862-9f34-40b1-9d1d-6c048c090abb None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:04.293682 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241a862-9f34-40b1-9d1d-6c048c090abb None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:04.295545 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:04.345877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9aea95-90c0-4a1a-bca9-c6b24be5f109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:04.346698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aea95-90c0-4a1a-bca9-c6b24be5f109 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:04.346991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aea95-90c0-4a1a-bca9-c6b24be5f109 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:04.347369 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aea95-90c0-4a1a-bca9-c6b24be5f109 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:04.348175 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9aea95-90c0-4a1a-bca9-c6b24be5f109 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:04.350588 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9aea95-90c0-4a1a-bca9-c6b24be5f109 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:04.359490 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 393/780] 10.222.0.78 () {56 vars in 1087 bytes} [Wed May 25 14:42:04 2022] GET /identity/v3/services => generated 772 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:04.611006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f23eaa-d2e1-477d-ac0c-42a09415720b None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:04.611410 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f23eaa-d2e1-477d-ac0c-42a09415720b None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:04.611892 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f23eaa-d2e1-477d-ac0c-42a09415720b None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:04.614428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 388/781] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:04.620303 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c45631-b855-4c8f-9390-32d6a76c1820 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:04.620639 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c45631-b855-4c8f-9390-32d6a76c1820 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:04.620882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c45631-b855-4c8f-9390-32d6a76c1820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:04.643697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-36c45631-b855-4c8f-9390-32d6a76c1820 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:04.644943 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-36c45631-b855-4c8f-9390-32d6a76c1820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:04.728821 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 394/782] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:04.753659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f23eaa-d2e1-477d-ac0c-42a09415720b None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:04.876029 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:04.939117 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f23eaa-d2e1-477d-ac0c-42a09415720b None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:04.990783 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fec745-4fa2-4887-bfbe-045370cf6f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:04.991585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fec745-4fa2-4887-bfbe-045370cf6f97 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:04.991865 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fec745-4fa2-4887-bfbe-045370cf6f97 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:04.992140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fec745-4fa2-4887-bfbe-045370cf6f97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:05.009855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fec745-4fa2-4887-bfbe-045370cf6f97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:42:05.018385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-08fec745-4fa2-4887-bfbe-045370cf6f97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:42:05.018385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:05.018385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:05.018385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:05.018385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:05.018385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:05.018385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:05.018385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:05.018385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:05.018385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:05.018385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:05.018385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:05.018385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:42:05.018385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:42:05.018385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:42:05.018385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:42:05.019360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:42:05.019360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:42:05.019360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:42:05.019360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:42:05.019360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:05.019360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:05.019360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:05.019360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:05.019360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:05.019360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:05.019360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:05.019360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:05.019360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:05.019360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:05.019360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:05.019360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:05.020206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:05.020206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:05.020206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:05.020206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:05.020206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:42:05.020206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:42:05.020206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:42:05.020206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:42:05.020206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:42:05.020206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:42:05.020206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:42:05.020206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:05.020206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 389/783] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:42:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:05.026252 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c45631-b855-4c8f-9390-32d6a76c1820 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:05.192211 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:05.200445 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c45631-b855-4c8f-9390-32d6a76c1820 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:05.251647 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c4331-6f64-4472-acf4-cb6fd7483f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:05.252336 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c4331-6f64-4472-acf4-cb6fd7483f3f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:05.252623 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c4331-6f64-4472-acf4-cb6fd7483f3f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:05.252904 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c4331-6f64-4472-acf4-cb6fd7483f3f None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:05.253657 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c4331-6f64-4472-acf4-cb6fd7483f3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:05.253966 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c4331-6f64-4472-acf4-cb6fd7483f3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:05.255375 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c4331-6f64-4472-acf4-cb6fd7483f3f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:05.267545 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 395/784] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:42:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:05.273193 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fec745-4fa2-4887-bfbe-045370cf6f97 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:05.402893 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fec745-4fa2-4887-bfbe-045370cf6f97 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:05.434985 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:05.447807 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c82ea9-1761-40bc-a394-3ef9d8b6527e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:05.448971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c82ea9-1761-40bc-a394-3ef9d8b6527e None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:05.449537 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c82ea9-1761-40bc-a394-3ef9d8b6527e None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:05.450055 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c82ea9-1761-40bc-a394-3ef9d8b6527e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:05.462982 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c82ea9-1761-40bc-a394-3ef9d8b6527e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:05.464927 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c82ea9-1761-40bc-a394-3ef9d8b6527e None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:05.484592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-16c82ea9-1761-40bc-a394-3ef9d8b6527e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:05.484592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:05.484592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:05.484592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:05.484592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:05.484592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:05.484592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:05.484592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:05.484592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:05.484592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:05.484592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:05.484592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:05.484592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:05.484592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:05.484592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:05.485566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:05.485566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:05.485566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:05.485566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:05.485566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:05.485566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:05.485566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:05.485566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:05.485566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:05.485566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:05.485566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:05.485566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:05.485566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:05.485566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:05.485566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:05.485566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:05.485566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:05.486495 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:05.486495 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:05.486495 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:05.486495 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:05.489003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 390/785] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:05.494559 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c4331-6f64-4472-acf4-cb6fd7483f3f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:05.648955 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c4331-6f64-4472-acf4-cb6fd7483f3f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:05.698928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8383f-c134-4559-bc87-9ca510f77485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:05.699528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8383f-c134-4559-bc87-9ca510f77485 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:05.699812 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8383f-c134-4559-bc87-9ca510f77485 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:05.700091 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8383f-c134-4559-bc87-9ca510f77485 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:05.700846 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8383f-c134-4559-bc87-9ca510f77485 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:05.701155 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8383f-c134-4559-bc87-9ca510f77485 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:05.702581 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8383f-c134-4559-bc87-9ca510f77485 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:05.715040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79977d5e-7b9e-431e-96a3-62111c93f290 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:05.715687 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79977d5e-7b9e-431e-96a3-62111c93f290 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:05.716833 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79977d5e-7b9e-431e-96a3-62111c93f290 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:05.718600 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 391/786] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:05.724035 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35533e74-f826-46a6-aab0-2371f2069ba0 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:05.724315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35533e74-f826-46a6-aab0-2371f2069ba0 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:05.724672 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35533e74-f826-46a6-aab0-2371f2069ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:05.727134 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 396/787] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:05.734100 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8383f-c134-4559-bc87-9ca510f77485 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:05.763110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-35533e74-f826-46a6-aab0-2371f2069ba0 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:05.764894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-35533e74-f826-46a6-aab0-2371f2069ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:05.778436 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:05.871793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8383f-c134-4559-bc87-9ca510f77485 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:05.913654 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993a62a-b6e0-49a2-bc6b-616c896186c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:05.913654 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993a62a-b6e0-49a2-bc6b-616c896186c2 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:05.913654 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993a62a-b6e0-49a2-bc6b-616c896186c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:05.913654 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993a62a-b6e0-49a2-bc6b-616c896186c2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:05.916395 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993a62a-b6e0-49a2-bc6b-616c896186c2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:05.917940 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993a62a-b6e0-49a2-bc6b-616c896186c2 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:05.941136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-0993a62a-b6e0-49a2-bc6b-616c896186c2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 14:42:05.941136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:05.941136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:05.941136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:05.941136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:05.941136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:05.941136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:05.941136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:05.941136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:05.941136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:05.941136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:05.941136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:05.941136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:42:05.941136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:42:05.941136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:42:05.941136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:42:05.942361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:05.942361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:05.942361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:42:05.942361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:05.942361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:42:05.942361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:05.942361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:05.942361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:05.942361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:05.942361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:05.942361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:05.942361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:05.942361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:05.942361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:05.942361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:05.942361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:05.942361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:05.942361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:05.943616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:05.943616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:05.943616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:42:05.943616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:42:05.943616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:42:05.943616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:42:05.943616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:42:05.943616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:42:05.943616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 14:42:05.943616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:05.943616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 397/788] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:42:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:05.948603 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993a62a-b6e0-49a2-bc6b-616c896186c2 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:05.964651 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 392/789] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:42:05.979621 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5c716-0f47-4226-9e8e-e1c1d5e038d5 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:05.980012 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5c716-0f47-4226-9e8e-e1c1d5e038d5 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:05.980439 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5c716-0f47-4226-9e8e-e1c1d5e038d5 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:05.983043 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 393/790] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:05.989465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3aa6b2-2ade-47da-b134-51cd04d3a1f1 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:05.989785 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3aa6b2-2ade-47da-b134-51cd04d3a1f1 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:05.990175 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3aa6b2-2ade-47da-b134-51cd04d3a1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:06.013859 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-9d3aa6b2-2ade-47da-b134-51cd04d3a1f1 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:06.015282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-9d3aa6b2-2ade-47da-b134-51cd04d3a1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:06.037432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:06.061823 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993a62a-b6e0-49a2-bc6b-616c896186c2 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:06.098127 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 394/791] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:06.099212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3aa6b2-2ade-47da-b134-51cd04d3a1f1 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:06.105049 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fb7b6-db52-4f2e-80cf-2585e7aaf70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:06.105775 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fb7b6-db52-4f2e-80cf-2585e7aaf70b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:06.106081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fb7b6-db52-4f2e-80cf-2585e7aaf70b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:06.106364 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fb7b6-db52-4f2e-80cf-2585e7aaf70b None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:06.107368 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3fb7b6-db52-4f2e-80cf-2585e7aaf70b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:06.107689 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3fb7b6-db52-4f2e-80cf-2585e7aaf70b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:06.109205 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3fb7b6-db52-4f2e-80cf-2585e7aaf70b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:06.131931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 398/792] 10.222.0.78 () {56 vars in 1135 bytes} [Wed May 25 14:42:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:06.137688 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fb7b6-db52-4f2e-80cf-2585e7aaf70b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:06.171250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:06.237551 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3aa6b2-2ade-47da-b134-51cd04d3a1f1 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:06.255269 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fb7b6-db52-4f2e-80cf-2585e7aaf70b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:06.282523 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed461f5b-bab1-429d-bc34-182234b214f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:06.283452 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed461f5b-bab1-429d-bc34-182234b214f1 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:06.283843 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed461f5b-bab1-429d-bc34-182234b214f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:06.284123 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed461f5b-bab1-429d-bc34-182234b214f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:06.298378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed461f5b-bab1-429d-bc34-182234b214f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:42:06.306753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-ed461f5b-bab1-429d-bc34-182234b214f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:42:06.306753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:06.306753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:06.306753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:06.306753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:06.306753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:06.306753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:06.306753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:06.306753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:06.306753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:06.306753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:06.306753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:06.306753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:42:06.306753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:42:06.306753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:42:06.306753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:42:06.308962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:42:06.308962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:42:06.308962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:42:06.308962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:42:06.308962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:06.308962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:06.308962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:06.308962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:06.308962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:06.308962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:06.308962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:06.308962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:06.308962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:06.308962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:06.308962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:06.308962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:06.311429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:06.311429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:06.311429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:06.311429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:06.311429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:42:06.311429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:42:06.311429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:42:06.311429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:42:06.311429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:42:06.311429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:42:06.311429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:42:06.311429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:06.318096 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 395/793] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:42:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:06.318096 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed461f5b-bab1-429d-bc34-182234b214f1 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:06.324502 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec93f4-aaa6-43a4-8dd8-9b96a8e8f0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:06.326236 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec93f4-aaa6-43a4-8dd8-9b96a8e8f0bd None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:06.327582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec93f4-aaa6-43a4-8dd8-9b96a8e8f0bd None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:06.328157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec93f4-aaa6-43a4-8dd8-9b96a8e8f0bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:06.333603 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:06.350592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec93f4-aaa6-43a4-8dd8-9b96a8e8f0bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:42:06.366360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-06ec93f4-aaa6-43a4-8dd8-9b96a8e8f0bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:42:06.366360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:06.366360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:06.366360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:06.366360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:06.366360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:06.366360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:06.366360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:06.366360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:06.366360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:06.366360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:06.366360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:06.366360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:42:06.366360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:42:06.366360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:42:06.366360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:42:06.368487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:42:06.368487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:42:06.368487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:42:06.368487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:42:06.368487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:06.368487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:06.368487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:06.368487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:06.368487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:06.368487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:06.368487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:06.368487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:06.368487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:06.368487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:06.368487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:06.368487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:06.370489 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:06.370489 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:06.370489 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:06.370489 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:06.370489 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:42:06.370489 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:42:06.370489 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:42:06.370489 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:42:06.370489 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:42:06.370489 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:42:06.370489 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:42:06.370489 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:06.378405 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 399/794] 10.222.0.78 () {56 vars in 1102 bytes} [Wed May 25 14:42:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:06.380105 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec93f4-aaa6-43a4-8dd8-9b96a8e8f0bd None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:06.509988 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed461f5b-bab1-429d-bc34-182234b214f1 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:06.515035 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:06.573101 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec93f4-aaa6-43a4-8dd8-9b96a8e8f0bd None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:06.576489 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79b07c-e551-448b-b5e6-05db1316f540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:06.577971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79b07c-e551-448b-b5e6-05db1316f540 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:06.578566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79b07c-e551-448b-b5e6-05db1316f540 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:06.580129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79b07c-e551-448b-b5e6-05db1316f540 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:06.599155 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf79b07c-e551-448b-b5e6-05db1316f540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:06.602104 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf79b07c-e551-448b-b5e6-05db1316f540 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:06.635629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-bf79b07c-e551-448b-b5e6-05db1316f540 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:06.635629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:06.635629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:06.635629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:06.635629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:06.635629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:06.635629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:06.635629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:06.635629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:06.635629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:06.635629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:06.635629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:06.635629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:06.635629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:06.635629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:06.637455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:06.637455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:06.637455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:06.637455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:06.637455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:06.637455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:06.637455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:06.637455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:06.637455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:06.637455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:06.637455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:06.637455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:06.637455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:06.637455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:06.637455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:06.637455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:06.637455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:06.639784 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:06.639784 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:06.639784 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:06.639784 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:06.644698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 396/795] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:06.646289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79b07c-e551-448b-b5e6-05db1316f540 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:06.650424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca5fb90-9520-4955-b592-fd98bb64dcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:06.652045 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5fb90-9520-4955-b592-fd98bb64dcb6 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:06.652871 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5fb90-9520-4955-b592-fd98bb64dcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:06.653615 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5fb90-9520-4955-b592-fd98bb64dcb6 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:06.655786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca5fb90-9520-4955-b592-fd98bb64dcb6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:06.656573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca5fb90-9520-4955-b592-fd98bb64dcb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:06.659316 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca5fb90-9520-4955-b592-fd98bb64dcb6 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:06.676090 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 400/796] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:42:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:06.677981 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca5fb90-9520-4955-b592-fd98bb64dcb6 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:06.772628 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:06.774796 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79b07c-e551-448b-b5e6-05db1316f540 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:06.813909 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f512d0-b3f6-4e77-8765-51810c3dfe95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:06.814601 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f512d0-b3f6-4e77-8765-51810c3dfe95 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:06.814863 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f512d0-b3f6-4e77-8765-51810c3dfe95 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:06.815136 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f512d0-b3f6-4e77-8765-51810c3dfe95 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:06.815886 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f512d0-b3f6-4e77-8765-51810c3dfe95 None admin] RBAC: Adding query filter params (name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:06.816195 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f512d0-b3f6-4e77-8765-51810c3dfe95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:06.817693 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f512d0-b3f6-4e77-8765-51810c3dfe95 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:06.833256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 397/797] 10.222.0.78 () {56 vars in 1103 bytes} [Wed May 25 14:42:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:06.833256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f512d0-b3f6-4e77-8765-51810c3dfe95 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:06.846502 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca5fb90-9520-4955-b592-fd98bb64dcb6 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:06.864337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:06.893531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ff2e7-b9ce-433c-9775-c0ce0368c083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:06.894487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ff2e7-b9ce-433c-9775-c0ce0368c083 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:06.894883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ff2e7-b9ce-433c-9775-c0ce0368c083 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:06.895261 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ff2e7-b9ce-433c-9775-c0ce0368c083 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:06.896296 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ff2e7-b9ce-433c-9775-c0ce0368c083 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:06.896724 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ff2e7-b9ce-433c-9775-c0ce0368c083 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:06.898811 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ff2e7-b9ce-433c-9775-c0ce0368c083 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:06.917777 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 401/798] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:06.919090 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ff2e7-b9ce-433c-9775-c0ce0368c083 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:06.965884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f512d0-b3f6-4e77-8765-51810c3dfe95 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:07.000352 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53cff3-706e-4255-aa01-c16acd545958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:07.001143 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53cff3-706e-4255-aa01-c16acd545958 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.001416 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53cff3-706e-4255-aa01-c16acd545958 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.001684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53cff3-706e-4255-aa01-c16acd545958 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:07.014548 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53cff3-706e-4255-aa01-c16acd545958 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:07.016755 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53cff3-706e-4255-aa01-c16acd545958 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:07.039196 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-fb53cff3-706e-4255-aa01-c16acd545958 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:07.039196 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:07.039196 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:07.039196 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:07.039196 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:07.039196 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:07.039196 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:07.039196 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:07.039196 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:07.039196 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:07.039196 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:07.039196 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:07.039196 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:07.039196 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:07.039196 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:07.040334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:07.040334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:07.040334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:07.040334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:07.040334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:07.040334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:07.040334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:07.040334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:07.040334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:07.040334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:07.040334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:07.040334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:07.040334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:07.040334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:07.040334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:07.040334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:07.040334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:07.041240 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:07.041240 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:07.041240 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:07.041240 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:07.041575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 398/799] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:07.043192 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f9921-5e51-4f47-9d1d-78fa951a2b5f None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.043585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f9921-5e51-4f47-9d1d-78fa951a2b5f None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.043958 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f9921-5e51-4f47-9d1d-78fa951a2b5f None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:07.050315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 399/800] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:07.051556 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f9921-5e51-4f47-9d1d-78fa951a2b5f None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:07.072578 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ff2e7-b9ce-433c-9775-c0ce0368c083 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:07.093029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:07.114184 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a83083-b4ab-404c-ae4c-fd0a22ec2ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:07.115053 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a83083-b4ab-404c-ae4c-fd0a22ec2ab6 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.115515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a83083-b4ab-404c-ae4c-fd0a22ec2ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.115801 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a83083-b4ab-404c-ae4c-fd0a22ec2ab6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:07.128518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a83083-b4ab-404c-ae4c-fd0a22ec2ab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:07.130056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a83083-b4ab-404c-ae4c-fd0a22ec2ab6 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:07.150671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-c6a83083-b4ab-404c-ae4c-fd0a22ec2ab6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:07.150671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:07.150671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:07.150671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:07.150671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:07.150671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:07.150671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:07.150671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:07.150671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:07.150671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:07.150671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:07.150671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:07.150671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:07.150671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:07.150671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:07.151733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:07.151733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:07.151733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:07.151733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:07.151733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:07.151733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:07.151733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:07.151733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:07.151733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:07.151733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:07.151733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:07.151733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:07.151733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:07.151733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:07.151733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:07.151733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:07.151733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:07.152639 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:07.152639 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:07.152639 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:07.152639 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:07.152639 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 402/801] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:07.152911 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a83083-b4ab-404c-ae4c-fd0a22ec2ab6 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:07.158714 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:07.198374 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f9921-5e51-4f47-9d1d-78fa951a2b5f None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:07.237408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb8cfd-ff28-42dd-9134-a97a6d7dd62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:07.238587 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb8cfd-ff28-42dd-9134-a97a6d7dd62a None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.238872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb8cfd-ff28-42dd-9134-a97a6d7dd62a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.239146 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb8cfd-ff28-42dd-9134-a97a6d7dd62a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:07.246625 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb8cfd-ff28-42dd-9134-a97a6d7dd62a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:07.248113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb8cfd-ff28-42dd-9134-a97a6d7dd62a None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:07.264182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-0cdb8cfd-ff28-42dd-9134-a97a6d7dd62a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:07.264182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:07.264182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:07.264182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:07.264182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:07.264182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:07.264182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:07.264182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:07.264182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:07.264182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:07.264182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:07.264182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:07.264182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:42:07.264182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:42:07.264182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:42:07.265110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:42:07.265110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:07.265110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:07.265110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:07.265110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:07.265110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:07.265110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:07.265110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:07.265110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:07.265110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:07.265110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:07.265110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:07.265110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:07.265110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:07.265110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:07.265110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:07.265110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:07.265994 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:42:07.265994 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:42:07.265994 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:42:07.265994 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:42:07.265994 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:42:07.265994 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:42:07.265994 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:07.265994 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:07.265994 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 400/802] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:42:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:07.266917 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e80201-c722-40ad-9316-dbfca82ea30b None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.266985 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e80201-c722-40ad-9316-dbfca82ea30b None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.267235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e80201-c722-40ad-9316-dbfca82ea30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:07.284914 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-79e80201-c722-40ad-9316-dbfca82ea30b None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:07.285960 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a83083-b4ab-404c-ae4c-fd0a22ec2ab6 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:07.286363 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-79e80201-c722-40ad-9316-dbfca82ea30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:07.323430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650f55c-2ae1-487c-a6c0-acca3749f3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:07.324010 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650f55c-2ae1-487c-a6c0-acca3749f3c1 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.324293 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650f55c-2ae1-487c-a6c0-acca3749f3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.324569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650f55c-2ae1-487c-a6c0-acca3749f3c1 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:07.325330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650f55c-2ae1-487c-a6c0-acca3749f3c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:07.325646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650f55c-2ae1-487c-a6c0-acca3749f3c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:07.327070 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650f55c-2ae1-487c-a6c0-acca3749f3c1 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:07.338556 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 403/803] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:07.339404 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650f55c-2ae1-487c-a6c0-acca3749f3c1 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:07.362173 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 401/804] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:07.363036 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e80201-c722-40ad-9316-dbfca82ea30b None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:07.380066 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:07.423948 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:07.456054 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650f55c-2ae1-487c-a6c0-acca3749f3c1 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:07.466458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e80201-c722-40ad-9316-dbfca82ea30b None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:07.492722 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf21bd8-e2ab-4e00-849e-95805a131e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:07.493385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf21bd8-e2ab-4e00-849e-95805a131e97 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.493666 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf21bd8-e2ab-4e00-849e-95805a131e97 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.493940 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf21bd8-e2ab-4e00-849e-95805a131e97 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:07.494686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf21bd8-e2ab-4e00-849e-95805a131e97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:07.494992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf21bd8-e2ab-4e00-849e-95805a131e97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:07.496365 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf21bd8-e2ab-4e00-849e-95805a131e97 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:07.503221 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209807c-d24e-4e18-b4f0-02697a9fad56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:07.503874 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209807c-d24e-4e18-b4f0-02697a9fad56 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.504151 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209807c-d24e-4e18-b4f0-02697a9fad56 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.504419 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209807c-d24e-4e18-b4f0-02697a9fad56 None admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:07.505205 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6209807c-d24e-4e18-b4f0-02697a9fad56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:07.505584 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6209807c-d24e-4e18-b4f0-02697a9fad56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:07.506955 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6209807c-d24e-4e18-b4f0-02697a9fad56 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:07.509022 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 404/805] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:07.510380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b98ba2-0693-46ad-8643-7fa05796982e None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.510668 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b98ba2-0693-46ad-8643-7fa05796982e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.510949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b98ba2-0693-46ad-8643-7fa05796982e None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:07.512165 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 405/806] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:07.512845 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b98ba2-0693-46ad-8643-7fa05796982e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:07.523926 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 402/807] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:42:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:07.525332 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209807c-d24e-4e18-b4f0-02697a9fad56 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:07.624876 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:07.636652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b98ba2-0693-46ad-8643-7fa05796982e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:07.638826 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209807c-d24e-4e18-b4f0-02697a9fad56 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:07.660889 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:07.677703 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191377f9-e87c-459e-a3ec-3afe5a801068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:07.678551 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191377f9-e87c-459e-a3ec-3afe5a801068 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.678843 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191377f9-e87c-459e-a3ec-3afe5a801068 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.679117 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191377f9-e87c-459e-a3ec-3afe5a801068 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:07.679697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f892f453-7952-4692-8c8d-27c11d209315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:07.679806 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191377f9-e87c-459e-a3ec-3afe5a801068 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:07.680623 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892f453-7952-4692-8c8d-27c11d209315 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.680905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892f453-7952-4692-8c8d-27c11d209315 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.681148 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191377f9-e87c-459e-a3ec-3afe5a801068 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:07.681273 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892f453-7952-4692-8c8d-27c11d209315 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:07.689519 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f892f453-7952-4692-8c8d-27c11d209315 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:07.691012 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f892f453-7952-4692-8c8d-27c11d209315 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:07.695005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-191377f9-e87c-459e-a3ec-3afe5a801068 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:42:07.695005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:07.695005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:42:07.695005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:42:07.695005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:42:07.695005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:42:07.695005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:42:07.695005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:07.695005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:42:07.695005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:07.695005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:42:07.695005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:07.695005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:07.695005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:07.695005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:07.695005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:07.696923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:07.696923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:07.696923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:07.696923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:07.696923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:07.696923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:07.696923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:07.696923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:42:07.696923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:42:07.696923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:42:07.696923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:42:07.696923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:07.696923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:07.696923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:07.696923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:07.696923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:07.698112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:07.698112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:07.698112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:07.698112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:07.698112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:07.698112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:07.698112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:07.698112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:07.698112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:07.698112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:07.698112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:07.698112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:42:07.698112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:07.698112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:42:07.698112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:07.698112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 403/808] 10.222.0.78 () {56 vars in 1129 bytes} [Wed May 25 14:42:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:07.699258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191377f9-e87c-459e-a3ec-3afe5a801068 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:07.707607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-f892f453-7952-4692-8c8d-27c11d209315 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:42:07.707607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:07.707607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:07.707607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:07.707607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:07.707607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:07.707607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:07.707607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:07.707607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:07.707607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:07.707607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:07.707607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:07.707607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:42:07.707607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:42:07.707607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:42:07.707607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:42:07.708987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:07.708987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:07.708987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:42:07.708987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:07.708987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:42:07.708987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:07.708987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:07.708987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:07.708987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:07.708987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:07.708987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:07.708987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:07.708987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:07.708987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:07.708987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:07.708987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:07.708987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:07.708987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:07.710313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:07.710313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:07.710313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:42:07.710313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:42:07.710313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:42:07.710313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:42:07.710313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:42:07.710313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:42:07.710313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:42:07.710313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:07.710313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 406/809] 10.222.0.78 () {56 vars in 1093 bytes} [Wed May 25 14:42:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:07.710313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c64d9-dab4-4a07-a695-9deba15df607 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.711279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c64d9-dab4-4a07-a695-9deba15df607 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.711279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c64d9-dab4-4a07-a695-9deba15df607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:07.731292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-e21c64d9-dab4-4a07-a695-9deba15df607 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:07.732641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e21c64d9-dab4-4a07-a695-9deba15df607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:07.809339 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 407/810] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:07.810319 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c64d9-dab4-4a07-a695-9deba15df607 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:07.842247 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191377f9-e87c-459e-a3ec-3afe5a801068 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:07.882240 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78084d64-4cfb-4ed5-a1aa-cd5b0662a900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:07.883041 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78084d64-4cfb-4ed5-a1aa-cd5b0662a900 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.883315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78084d64-4cfb-4ed5-a1aa-cd5b0662a900 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.883583 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78084d64-4cfb-4ed5-a1aa-cd5b0662a900 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:07.895830 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78084d64-4cfb-4ed5-a1aa-cd5b0662a900 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:07.897857 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78084d64-4cfb-4ed5-a1aa-cd5b0662a900 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:07.899888 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:07.915458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-78084d64-4cfb-4ed5-a1aa-cd5b0662a900 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:42:07.915458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:07.915458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:07.915458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:07.915458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:07.915458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:07.915458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:07.915458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:07.915458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:07.915458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:07.915458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:07.915458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:07.915458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:42:07.915458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:42:07.915458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:42:07.915458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:42:07.917243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:07.917243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:07.917243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:42:07.917243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:07.917243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:42:07.917243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:07.917243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:07.917243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:07.917243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:07.917243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:07.917243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:07.917243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:07.917243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:07.917243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:07.917243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:07.917243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:07.917243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:07.917243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:07.919376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:07.919376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:07.919376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:42:07.919376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:42:07.919376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:42:07.919376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:42:07.919376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:42:07.919376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:42:07.919376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:42:07.919376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:07.919376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 404/811] 10.222.0.78 () {56 vars in 1117 bytes} [Wed May 25 14:42:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:07.919376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78084d64-4cfb-4ed5-a1aa-cd5b0662a900 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:07.928569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c64d9-dab4-4a07-a695-9deba15df607 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:07.968822 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c217044-0459-4211-80b9-bfffd395dc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:07.970311 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c217044-0459-4211-80b9-bfffd395dc01 None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.970779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c217044-0459-4211-80b9-bfffd395dc01 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.971241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c217044-0459-4211-80b9-bfffd395dc01 None admin] PATH_INFO: `/v3/projects/908a13d2b49544fbaf760b07e213bb7d/users/fbe4a2d106d84fc9ac65c046ba675353/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:07.974301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c217044-0459-4211-80b9-bfffd395dc01 None admin] RBAC: Authorizing `identity:create_grant(project_id=908a13d2b49544fbaf760b07e213bb7d, user_id=fbe4a2d106d84fc9ac65c046ba675353, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:07.976694 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c217044-0459-4211-80b9-bfffd395dc01 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:07.981524 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:07.994598 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 408/812] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:42:07 2022] PUT /identity/v3/projects/908a13d2b49544fbaf760b07e213bb7d/users/fbe4a2d106d84fc9ac65c046ba675353/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:07.995441 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c217044-0459-4211-80b9-bfffd395dc01 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:08.075002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78084d64-4cfb-4ed5-a1aa-cd5b0662a900 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:08.119788 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0806d6-a6c8-4d27-9a6d-e2f3d2050b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:08.120471 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0806d6-a6c8-4d27-9a6d-e2f3d2050b07 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:08.120749 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0806d6-a6c8-4d27-9a6d-e2f3d2050b07 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:08.121022 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0806d6-a6c8-4d27-9a6d-e2f3d2050b07 None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:08.121741 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0806d6-a6c8-4d27-9a6d-e2f3d2050b07 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:08.122090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0806d6-a6c8-4d27-9a6d-e2f3d2050b07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:08.123542 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0806d6-a6c8-4d27-9a6d-e2f3d2050b07 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:08.133868 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 405/813] 10.222.0.78 () {56 vars in 1124 bytes} [Wed May 25 14:42:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:08.133868 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0806d6-a6c8-4d27-9a6d-e2f3d2050b07 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:08.134156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c217044-0459-4211-80b9-bfffd395dc01 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:08.201367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006154b-a734-4e62-8077-e2804a6f3d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:08.202108 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006154b-a734-4e62-8077-e2804a6f3d46 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:08.202394 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006154b-a734-4e62-8077-e2804a6f3d46 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:08.202759 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006154b-a734-4e62-8077-e2804a6f3d46 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:08.203698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006154b-a734-4e62-8077-e2804a6f3d46 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:08.204058 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006154b-a734-4e62-8077-e2804a6f3d46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:08.205505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006154b-a734-4e62-8077-e2804a6f3d46 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:08.216846 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:08.228725 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 409/814] 10.222.0.78 () {56 vars in 1133 bytes} [Wed May 25 14:42:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:08.229755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006154b-a734-4e62-8077-e2804a6f3d46 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:08.274837 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0806d6-a6c8-4d27-9a6d-e2f3d2050b07 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:08.286209 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:08.317347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6936d-cdad-4718-9265-d979a97f4190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:08.318182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6936d-cdad-4718-9265-d979a97f4190 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:08.318564 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6936d-cdad-4718-9265-d979a97f4190 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:08.318839 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6936d-cdad-4718-9265-d979a97f4190 None admin] PATH_INFO: `/v3/services/network` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:08.319497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6936d-cdad-4718-9265-d979a97f4190 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:08.321277 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6936d-cdad-4718-9265-d979a97f4190 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:08.333708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-17f6936d-cdad-4718-9265-d979a97f4190 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:42:08.333708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:08.333708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:42:08.333708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:42:08.333708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:42:08.333708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:42:08.333708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:42:08.333708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:08.333708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:42:08.333708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:08.333708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:42:08.333708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:08.333708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:08.333708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:08.333708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:08.333708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:08.333708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:08.335054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:08.335054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:08.335054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:08.335054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:08.335054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:08.335054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:08.335054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:42:08.335054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:42:08.335054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:42:08.335054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:42:08.335054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:08.335054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:08.335054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:08.335054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:08.335054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:08.335054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:08.336170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:08.336170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:08.336170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:08.336170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:08.336170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:08.336170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:08.336170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:08.336170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:08.336170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:08.336170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:08.336170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:42:08.336170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:08.336170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:42:08.336170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:08.336170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 406/815] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:42:08 2022] GET /identity/v3/services/network => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:08.336986 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6936d-cdad-4718-9265-d979a97f4190 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:08.353415 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006154b-a734-4e62-8077-e2804a6f3d46 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:08.391075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc846115-370e-4914-b3a9-89afafbc1acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:08.391684 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc846115-370e-4914-b3a9-89afafbc1acc None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:08.391958 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc846115-370e-4914-b3a9-89afafbc1acc None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:08.392241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc846115-370e-4914-b3a9-89afafbc1acc None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:08.393086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc846115-370e-4914-b3a9-89afafbc1acc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:08.393400 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc846115-370e-4914-b3a9-89afafbc1acc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:08.394891 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc846115-370e-4914-b3a9-89afafbc1acc None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:08.420107 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 410/816] 10.222.0.78 () {56 vars in 1149 bytes} [Wed May 25 14:42:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:08.420107 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc846115-370e-4914-b3a9-89afafbc1acc None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:08.458474 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6936d-cdad-4718-9265-d979a97f4190 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:08.485371 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:08.499874 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92430f87-e242-4569-bb35-de38380837a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:08.500543 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92430f87-e242-4569-bb35-de38380837a6 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:08.500824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92430f87-e242-4569-bb35-de38380837a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:08.501123 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92430f87-e242-4569-bb35-de38380837a6 None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:08.501835 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92430f87-e242-4569-bb35-de38380837a6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:08.502181 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92430f87-e242-4569-bb35-de38380837a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:08.503603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92430f87-e242-4569-bb35-de38380837a6 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:08.508904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 407/817] 10.222.0.78 () {56 vars in 1124 bytes} [Wed May 25 14:42:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:08.509732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92430f87-e242-4569-bb35-de38380837a6 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:08.547808 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc846115-370e-4914-b3a9-89afafbc1acc None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:08.560599 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:08.586386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea4ad0-c676-4289-b2ad-152f0e9a7235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:08.587057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea4ad0-c676-4289-b2ad-152f0e9a7235 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:08.587493 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea4ad0-c676-4289-b2ad-152f0e9a7235 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:08.587780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea4ad0-c676-4289-b2ad-152f0e9a7235 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:08.638731 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92430f87-e242-4569-bb35-de38380837a6 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:08.671531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ea4ad0-c676-4289-b2ad-152f0e9a7235 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:08.674257 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ea4ad0-c676-4289-b2ad-152f0e9a7235 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:08.678080 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14012659-be08-4fd4-a3fe-f2b964184d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:08.678776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14012659-be08-4fd4-a3fe-f2b964184d5b None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:08.679036 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14012659-be08-4fd4-a3fe-f2b964184d5b None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:08.679308 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14012659-be08-4fd4-a3fe-f2b964184d5b None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:08.680022 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14012659-be08-4fd4-a3fe-f2b964184d5b None admin] RBAC: Adding query filter params (name=network) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:08.680331 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14012659-be08-4fd4-a3fe-f2b964184d5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:08.681694 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14012659-be08-4fd4-a3fe-f2b964184d5b None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:08.686736 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 408/818] 10.222.0.78 () {56 vars in 1112 bytes} [Wed May 25 14:42:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:08.687554 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14012659-be08-4fd4-a3fe-f2b964184d5b None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:08.759420 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:08.829725 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 411/819] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:42:08 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:42:08.829725 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea4ad0-c676-4289-b2ad-152f0e9a7235 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:08.831268 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:08.835150 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14012659-be08-4fd4-a3fe-f2b964184d5b None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:08.873955 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef694c2e-407b-4414-9451-593efe903a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:08.874807 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef694c2e-407b-4414-9451-593efe903a65 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:08.875089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef694c2e-407b-4414-9451-593efe903a65 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:08.875363 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef694c2e-407b-4414-9451-593efe903a65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:08.888805 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef694c2e-407b-4414-9451-593efe903a65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:08.890303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef694c2e-407b-4414-9451-593efe903a65 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:08.916977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-ef694c2e-407b-4414-9451-593efe903a65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:08.916977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:08.916977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:08.916977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:08.916977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:08.916977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:08.916977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:08.916977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:08.916977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:08.916977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:08.916977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:08.916977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:08.916977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:08.916977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:08.916977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:08.918316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:08.918316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:08.918316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:08.918316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:08.918316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:08.918316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:08.918316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:08.918316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:08.918316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:08.918316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:08.918316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:08.918316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:08.918316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:08.918316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:08.918316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:08.918316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:08.918316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:08.919233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:08.919233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:08.919233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:08.919233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:08.919929 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 409/820] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:08.920398 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef694c2e-407b-4414-9451-593efe903a65 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:08.951720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea4ad0-c676-4289-b2ad-152f0e9a7235 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:08.987729 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08ab2c-aa50-4778-9116-78aeb2c360a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:08.988415 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08ab2c-aa50-4778-9116-78aeb2c360a9 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:08.988681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08ab2c-aa50-4778-9116-78aeb2c360a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:08.988958 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08ab2c-aa50-4778-9116-78aeb2c360a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:08.989690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08ab2c-aa50-4778-9116-78aeb2c360a9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:08.991117 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08ab2c-aa50-4778-9116-78aeb2c360a9 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:09.017515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 412/821] 10.222.0.78 () {60 vars in 1144 bytes} [Wed May 25 14:42:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:42:09.018405 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08ab2c-aa50-4778-9116-78aeb2c360a9 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:09.048508 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:09.054690 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef694c2e-407b-4414-9451-593efe903a65 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:09.113288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ad541-294b-4e55-991e-1c424f41a2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:09.114339 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ad541-294b-4e55-991e-1c424f41a2fe None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:09.114803 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ad541-294b-4e55-991e-1c424f41a2fe None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:09.115288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ad541-294b-4e55-991e-1c424f41a2fe None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:09.116187 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ad541-294b-4e55-991e-1c424f41a2fe None admin] RBAC: Adding query filter params (type=network) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:09.116689 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ad541-294b-4e55-991e-1c424f41a2fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:09.118299 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ad541-294b-4e55-991e-1c424f41a2fe None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:09.125320 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 410/822] 10.222.0.78 () {56 vars in 1112 bytes} [Wed May 25 14:42:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:09.126629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ad541-294b-4e55-991e-1c424f41a2fe None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:09.147430 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:09.194985 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08ab2c-aa50-4778-9116-78aeb2c360a9 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:09.248818 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd92b1-90a2-4a4b-89ef-be4c3dca746f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:09.249846 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd92b1-90a2-4a4b-89ef-be4c3dca746f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:09.250222 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd92b1-90a2-4a4b-89ef-be4c3dca746f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:09.250453 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd92b1-90a2-4a4b-89ef-be4c3dca746f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:09.265614 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ad541-294b-4e55-991e-1c424f41a2fe None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:09.270983 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd92b1-90a2-4a4b-89ef-be4c3dca746f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:09.277545 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd92b1-90a2-4a4b-89ef-be4c3dca746f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:09.302957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-a1fd92b1-90a2-4a4b-89ef-be4c3dca746f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:09.302957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:09.302957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:09.302957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:09.302957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:09.302957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:09.302957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:09.302957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:09.302957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:09.302957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:09.302957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:09.302957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:09.302957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:09.302957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:09.302957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:09.303928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:09.303928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:09.303928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:09.303928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:09.303928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:09.303928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:09.303928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:09.303928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:09.303928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:09.303928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:09.303928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:09.303928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:09.303928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:09.303928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:09.303928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:09.303928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:09.303928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:09.304834 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:09.304834 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:09.304834 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:09.304834 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:09.304834 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 413/823] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:09.305688 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd92b1-90a2-4a4b-89ef-be4c3dca746f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:09.331865 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af342a-2cce-4e16-a9a6-8104a03a8f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:09.332640 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af342a-2cce-4e16-a9a6-8104a03a8f33 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:09.332927 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af342a-2cce-4e16-a9a6-8104a03a8f33 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:09.333257 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af342a-2cce-4e16-a9a6-8104a03a8f33 None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:09.334092 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99af342a-2cce-4e16-a9a6-8104a03a8f33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:09.334659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99af342a-2cce-4e16-a9a6-8104a03a8f33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:09.336284 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99af342a-2cce-4e16-a9a6-8104a03a8f33 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:09.357930 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 411/824] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:09.361150 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af342a-2cce-4e16-a9a6-8104a03a8f33 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:09.385353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:09.440839 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd92b1-90a2-4a4b-89ef-be4c3dca746f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:09.481647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:09.499422 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc05f9-3a54-4092-bbfc-5c8a4146bebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:09.500646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc05f9-3a54-4092-bbfc-5c8a4146bebf None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:09.501109 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc05f9-3a54-4092-bbfc-5c8a4146bebf None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:09.501947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc05f9-3a54-4092-bbfc-5c8a4146bebf None admin] PATH_INFO: `/v3/endpoints` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:09.502921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc05f9-3a54-4092-bbfc-5c8a4146bebf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:09.505245 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc05f9-3a54-4092-bbfc-5c8a4146bebf None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:09.518760 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af342a-2cce-4e16-a9a6-8104a03a8f33 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:09.536855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 414/825] 10.222.0.78 () {60 vars in 1144 bytes} [Wed May 25 14:42:09 2022] POST /identity/v3/endpoints => generated 327 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:42:09.538570 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc05f9-3a54-4092-bbfc-5c8a4146bebf None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:09.590188 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1d737-1734-4772-8ea7-3a56682d6b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:09.592148 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1d737-1734-4772-8ea7-3a56682d6b3e None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:09.592901 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1d737-1734-4772-8ea7-3a56682d6b3e None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:09.593532 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1d737-1734-4772-8ea7-3a56682d6b3e None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:09.595021 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1d737-1734-4772-8ea7-3a56682d6b3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:09.595810 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1d737-1734-4772-8ea7-3a56682d6b3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:09.598666 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1d737-1734-4772-8ea7-3a56682d6b3e None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:09.621895 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 412/826] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:09.627631 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1d737-1734-4772-8ea7-3a56682d6b3e None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:09.656826 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc05f9-3a54-4092-bbfc-5c8a4146bebf None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:09.719291 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:09.777831 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4fcdeb-64ba-409f-b31a-c1e15b310167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:09.778662 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4fcdeb-64ba-409f-b31a-c1e15b310167 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:09.779459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4fcdeb-64ba-409f-b31a-c1e15b310167 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:09.779741 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4fcdeb-64ba-409f-b31a-c1e15b310167 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:09.806968 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4fcdeb-64ba-409f-b31a-c1e15b310167 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:09.808478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4fcdeb-64ba-409f-b31a-c1e15b310167 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:09.810466 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1d737-1734-4772-8ea7-3a56682d6b3e None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:09.828126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-0a4fcdeb-64ba-409f-b31a-c1e15b310167 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:09.828126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:09.828126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:09.828126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:09.828126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:09.828126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:09.828126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:09.828126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:09.828126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:09.828126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:09.828126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:09.828126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:09.828126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:42:09.828126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:42:09.828126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:42:09.829539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:42:09.829539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:09.829539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:09.829539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:09.829539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:09.829539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:09.829539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:09.829539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:09.829539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:09.829539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:09.829539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:09.829539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:09.829539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:09.829539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:09.829539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:09.829539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:09.829539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:09.830972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:42:09.830972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:42:09.830972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:42:09.830972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:42:09.830972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:42:09.830972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:42:09.830972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:09.830972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:09.830972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 415/827] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:42:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:09.835333 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4fcdeb-64ba-409f-b31a-c1e15b310167 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:09.860510 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:09.875074 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175ffa6-ac04-4386-a60b-3f8f7acdf52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:09.875889 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175ffa6-ac04-4386-a60b-3f8f7acdf52c None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:09.876377 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175ffa6-ac04-4386-a60b-3f8f7acdf52c None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:09.876718 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175ffa6-ac04-4386-a60b-3f8f7acdf52c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:09.884784 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175ffa6-ac04-4386-a60b-3f8f7acdf52c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:09.886503 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175ffa6-ac04-4386-a60b-3f8f7acdf52c None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:09.904646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-3175ffa6-ac04-4386-a60b-3f8f7acdf52c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:09.904646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:09.904646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:09.904646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:09.904646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:09.904646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:09.904646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:09.904646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:09.904646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:09.904646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:09.904646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:09.904646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:09.904646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:42:09.904646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:42:09.904646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:42:09.905720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:42:09.905720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:09.905720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:09.905720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:09.905720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:09.905720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:09.905720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:09.905720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:09.905720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:09.905720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:09.905720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:09.905720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:09.905720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:09.905720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:09.905720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:09.905720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:09.905720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:09.906894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:42:09.906894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:42:09.906894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:42:09.906894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:42:09.906894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:42:09.906894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:42:09.906894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:09.906894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:09.906894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 413/828] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:42:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:09.915924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175ffa6-ac04-4386-a60b-3f8f7acdf52c None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:09.971982 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4fcdeb-64ba-409f-b31a-c1e15b310167 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:10.038492 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e66a62-8d39-43c6-b1fb-68d276d71d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:10.039847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e66a62-8d39-43c6-b1fb-68d276d71d92 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:10.040660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e66a62-8d39-43c6-b1fb-68d276d71d92 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:10.041175 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e66a62-8d39-43c6-b1fb-68d276d71d92 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:10.042800 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e66a62-8d39-43c6-b1fb-68d276d71d92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:10.043418 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e66a62-8d39-43c6-b1fb-68d276d71d92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:10.045645 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e66a62-8d39-43c6-b1fb-68d276d71d92 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:10.063270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 416/829] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:42:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:10.070062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e66a62-8d39-43c6-b1fb-68d276d71d92 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:10.096878 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175ffa6-ac04-4386-a60b-3f8f7acdf52c None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:10.129475 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:10.145007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291f3b5-c2b5-46c3-8e4e-4ab7b6728ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:10.145673 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291f3b5-c2b5-46c3-8e4e-4ab7b6728ea0 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:10.145955 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291f3b5-c2b5-46c3-8e4e-4ab7b6728ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:10.146275 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291f3b5-c2b5-46c3-8e4e-4ab7b6728ea0 None admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:10.147407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291f3b5-c2b5-46c3-8e4e-4ab7b6728ea0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:10.147718 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291f3b5-c2b5-46c3-8e4e-4ab7b6728ea0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:10.149976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291f3b5-c2b5-46c3-8e4e-4ab7b6728ea0 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:10.170126 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 414/830] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:42:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:10.174344 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291f3b5-c2b5-46c3-8e4e-4ab7b6728ea0 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:10.220742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e66a62-8d39-43c6-b1fb-68d276d71d92 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:10.235632 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:10.269335 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf59ae-bea5-4ed9-89a5-d7aaba41b17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:10.270613 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf59ae-bea5-4ed9-89a5-d7aaba41b17d None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:10.271084 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf59ae-bea5-4ed9-89a5-d7aaba41b17d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:10.271540 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf59ae-bea5-4ed9-89a5-d7aaba41b17d None admin] PATH_INFO: `/v3/projects/908a13d2b49544fbaf760b07e213bb7d/users/b2935579c8554bed8b1e211c47548ac5/roles/336f27e7f8624a30b30cef9acada97a0` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:10.293977 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf59ae-bea5-4ed9-89a5-d7aaba41b17d None admin] RBAC: Authorizing `identity:create_grant(project_id=908a13d2b49544fbaf760b07e213bb7d, user_id=b2935579c8554bed8b1e211c47548ac5, role_id=336f27e7f8624a30b30cef9acada97a0)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:10.296047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf59ae-bea5-4ed9-89a5-d7aaba41b17d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:10.322290 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291f3b5-c2b5-46c3-8e4e-4ab7b6728ea0 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:10.322492 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 417/831] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:42:10 2022] PUT /identity/v3/projects/908a13d2b49544fbaf760b07e213bb7d/users/b2935579c8554bed8b1e211c47548ac5/roles/336f27e7f8624a30b30cef9acada97a0 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:42:10.376707 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb9cb2-7326-490b-a104-898766eab759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:10.377640 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb9cb2-7326-490b-a104-898766eab759 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:10.377937 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb9cb2-7326-490b-a104-898766eab759 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:10.378244 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb9cb2-7326-490b-a104-898766eab759 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:10.379356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb9cb2-7326-490b-a104-898766eab759 None admin] RBAC: Adding query filter params (role.id=e566b37cd92f467198b5dc40f183effc, scope.project.id=908a13d2b49544fbaf760b07e213bb7d, user.id=e34e1d7ffbac49b0bde45829af0c6925) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:10.379829 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb9cb2-7326-490b-a104-898766eab759 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:10.382928 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb9cb2-7326-490b-a104-898766eab759 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:10.393024 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 415/832] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:42:10 2022] GET /identity/v3/role_assignments?user.id=e34e1d7ffbac49b0bde45829af0c6925&scope.project.id=908a13d2b49544fbaf760b07e213bb7d&role.id=e566b37cd92f467198b5dc40f183effc => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:10.761531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2341684-d717-4f21-8080-10f2c12ee658 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:10.762270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2341684-d717-4f21-8080-10f2c12ee658 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:10.762270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2341684-d717-4f21-8080-10f2c12ee658 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:10.766488 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 418/833] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:10.773508 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21542579-feef-4e4f-9137-0cae44a4fb92 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:10.773794 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21542579-feef-4e4f-9137-0cae44a4fb92 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:10.774105 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21542579-feef-4e4f-9137-0cae44a4fb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:10.792295 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-21542579-feef-4e4f-9137-0cae44a4fb92 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:10.793569 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-21542579-feef-4e4f-9137-0cae44a4fb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:10.889453 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 416/834] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:10.928487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2341684-d717-4f21-8080-10f2c12ee658 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:11.047542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:11.064771 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2341684-d717-4f21-8080-10f2c12ee658 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:11.103471 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3aa18-45f4-4fa3-b48c-35ec9245c97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:11.104328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3aa18-45f4-4fa3-b48c-35ec9245c97b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:11.104617 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3aa18-45f4-4fa3-b48c-35ec9245c97b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:11.104894 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3aa18-45f4-4fa3-b48c-35ec9245c97b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:11.120915 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3aa18-45f4-4fa3-b48c-35ec9245c97b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:42:11.129632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-02d3aa18-45f4-4fa3-b48c-35ec9245c97b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:42:11.129632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:11.129632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:11.129632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:11.129632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:11.129632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:11.129632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:11.129632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:11.129632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:11.129632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:11.129632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:11.129632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:11.129632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:42:11.129632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:42:11.129632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:42:11.129632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:42:11.131112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:42:11.131112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:42:11.131112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:42:11.131112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:42:11.131112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:11.131112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:11.131112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:11.131112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:11.131112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:11.131112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:11.131112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:11.131112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:11.131112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:11.131112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:11.131112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:11.131112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:11.132470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:11.132470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:11.132470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:11.132470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:11.132470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:42:11.132470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:42:11.132470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:42:11.132470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:42:11.132470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:42:11.132470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:42:11.132470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:42:11.132470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:11.132470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 419/835] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:42:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:11.136818 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21542579-feef-4e4f-9137-0cae44a4fb92 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:11.194880 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:11.261968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21542579-feef-4e4f-9137-0cae44a4fb92 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:11.300931 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a5353-5bfd-4cf8-a9ef-6afab8b839e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:11.301513 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a5353-5bfd-4cf8-a9ef-6afab8b839e1 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:11.301793 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a5353-5bfd-4cf8-a9ef-6afab8b839e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:11.302091 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a5353-5bfd-4cf8-a9ef-6afab8b839e1 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:11.302846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a5353-5bfd-4cf8-a9ef-6afab8b839e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:11.303148 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a5353-5bfd-4cf8-a9ef-6afab8b839e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:11.304536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a5353-5bfd-4cf8-a9ef-6afab8b839e1 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:11.314793 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 417/836] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:42:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:11.320706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3aa18-45f4-4fa3-b48c-35ec9245c97b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:11.439375 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3aa18-45f4-4fa3-b48c-35ec9245c97b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:11.475047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663c71a-5223-41c6-8e1b-4d373344b733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:11.475845 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663c71a-5223-41c6-8e1b-4d373344b733 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:11.476133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663c71a-5223-41c6-8e1b-4d373344b733 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:11.476413 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663c71a-5223-41c6-8e1b-4d373344b733 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:11.489051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3663c71a-5223-41c6-8e1b-4d373344b733 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:11.490565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3663c71a-5223-41c6-8e1b-4d373344b733 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:11.511214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-3663c71a-5223-41c6-8e1b-4d373344b733 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:11.511214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:11.511214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:11.511214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:11.511214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:11.511214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:11.511214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:11.511214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:11.511214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:11.511214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:11.511214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:11.511214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:11.511214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:11.511214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:11.511214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:11.513770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:11.513770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:11.513770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:11.513770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:11.513770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:11.513770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:11.513770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:11.513770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:11.513770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:11.513770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:11.513770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:11.513770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:11.513770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:11.513770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:11.513770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:11.513770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:11.513770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:11.515363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:11.515363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:11.515363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:11.515363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:11.515363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 420/837] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:11.519174 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a5353-5bfd-4cf8-a9ef-6afab8b839e1 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:11.603429 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1f984-cc01-478b-bd59-eae1c43815e5 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:11.603854 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1f984-cc01-478b-bd59-eae1c43815e5 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:11.603854 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1f984-cc01-478b-bd59-eae1c43815e5 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:11.605599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 421/838] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:11.611479 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c812afd-56ad-40bc-a870-adbf588f1376 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:11.611776 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c812afd-56ad-40bc-a870-adbf588f1376 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:11.612033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c812afd-56ad-40bc-a870-adbf588f1376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:11.618174 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:11.633878 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a5353-5bfd-4cf8-a9ef-6afab8b839e1 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:11.639967 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-3c812afd-56ad-40bc-a870-adbf588f1376 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:11.641733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3c812afd-56ad-40bc-a870-adbf588f1376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:11.646442 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:11.681014 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbfeb5a-f872-4dd4-bc36-a3423a01067d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:11.681685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfeb5a-f872-4dd4-bc36-a3423a01067d None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:11.681963 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfeb5a-f872-4dd4-bc36-a3423a01067d None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:11.682269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfeb5a-f872-4dd4-bc36-a3423a01067d None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:11.683291 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbfeb5a-f872-4dd4-bc36-a3423a01067d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:11.684538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbfeb5a-f872-4dd4-bc36-a3423a01067d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:11.686809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbfeb5a-f872-4dd4-bc36-a3423a01067d None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:11.701256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 418/839] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:11.706419 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbfeb5a-f872-4dd4-bc36-a3423a01067d None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:11.730158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 422/840] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:11 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:42:11.748392 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c812afd-56ad-40bc-a870-adbf588f1376 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:11.821292 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbfeb5a-f872-4dd4-bc36-a3423a01067d None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:11.868396 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1074c-0ed7-408e-97de-f001766e9859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:11.869392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1074c-0ed7-408e-97de-f001766e9859 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:11.869680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1074c-0ed7-408e-97de-f001766e9859 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:11.869952 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1074c-0ed7-408e-97de-f001766e9859 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:11.874664 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c812afd-56ad-40bc-a870-adbf588f1376 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:11.880259 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b1074c-0ed7-408e-97de-f001766e9859 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:11.881730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b1074c-0ed7-408e-97de-f001766e9859 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:11.904434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-38b1074c-0ed7-408e-97de-f001766e9859 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 14:42:11.904434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:11.904434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:11.904434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:11.904434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:11.904434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:11.904434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:11.904434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:11.904434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:11.904434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:11.904434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:11.904434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:11.904434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:42:11.904434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:42:11.904434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:42:11.904434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:42:11.905429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:11.905429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:11.905429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:42:11.905429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:11.905429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:42:11.905429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:11.905429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:11.905429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:11.905429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:11.905429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:11.905429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:11.905429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:11.905429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:11.905429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:11.905429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:11.905429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:11.905429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:11.905429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:11.906505 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:11.906505 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:11.906505 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:42:11.906505 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:42:11.906505 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:42:11.906505 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:42:11.906505 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:42:11.906505 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:42:11.906505 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 14:42:11.906505 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:11.911229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 419/841] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:42:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:11.912328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1074c-0ed7-408e-97de-f001766e9859 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:11.923222 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:11.929791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f687cc0-8b76-4c64-931d-d1d7c6013c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:11.931096 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f687cc0-8b76-4c64-931d-d1d7c6013c14 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:11.931596 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f687cc0-8b76-4c64-931d-d1d7c6013c14 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:11.932100 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f687cc0-8b76-4c64-931d-d1d7c6013c14 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:11.933011 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f687cc0-8b76-4c64-931d-d1d7c6013c14 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:11.935067 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f687cc0-8b76-4c64-931d-d1d7c6013c14 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:11.946322 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:11.951605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-3f687cc0-8b76-4c64-931d-d1d7c6013c14 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:42:11.951605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:11.951605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:42:11.951605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:42:11.951605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:42:11.951605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:42:11.951605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:42:11.951605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:11.951605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:42:11.951605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:11.951605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:42:11.951605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:11.951605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:11.951605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:11.951605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:11.951605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:11.951605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:11.952615 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:11.952615 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:11.952615 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:11.952615 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:11.952615 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:11.952615 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:11.952615 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:42:11.952615 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:42:11.952615 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:42:11.952615 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:42:11.952615 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:11.952615 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:11.952615 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:11.952615 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:11.952615 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:11.952615 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:11.953452 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:11.953452 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:11.953452 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:11.953452 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:11.953452 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:11.953452 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:11.953452 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:11.953452 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:11.953452 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:11.953452 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:11.953452 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:42:11.953452 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:11.953452 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:42:11.953452 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:11.960983 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 423/842] 10.222.0.78 () {56 vars in 1114 bytes} [Wed May 25 14:42:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:11.962165 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f687cc0-8b76-4c64-931d-d1d7c6013c14 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:12.046423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1074c-0ed7-408e-97de-f001766e9859 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:12.103811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544994c-edc7-48c3-8de0-21970808da53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:12.104842 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544994c-edc7-48c3-8de0-21970808da53 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:12.105335 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544994c-edc7-48c3-8de0-21970808da53 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:12.105897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544994c-edc7-48c3-8de0-21970808da53 None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:12.107024 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a544994c-edc7-48c3-8de0-21970808da53 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:12.107603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a544994c-edc7-48c3-8de0-21970808da53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:12.109347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a544994c-edc7-48c3-8de0-21970808da53 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:12.124251 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f687cc0-8b76-4c64-931d-d1d7c6013c14 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:12.137925 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 420/843] 10.222.0.78 () {56 vars in 1135 bytes} [Wed May 25 14:42:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:12.145436 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544994c-edc7-48c3-8de0-21970808da53 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:12.177451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02199e0-603f-4d93-a742-85a3675bc132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:12.178687 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02199e0-603f-4d93-a742-85a3675bc132 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:12.179087 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02199e0-603f-4d93-a742-85a3675bc132 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:12.179610 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02199e0-603f-4d93-a742-85a3675bc132 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:12.180527 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02199e0-603f-4d93-a742-85a3675bc132 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:12.180848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02199e0-603f-4d93-a742-85a3675bc132 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:12.182405 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02199e0-603f-4d93-a742-85a3675bc132 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:12.193066 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 424/844] 10.222.0.78 () {56 vars in 1114 bytes} [Wed May 25 14:42:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:12.197265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02199e0-603f-4d93-a742-85a3675bc132 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:12.268655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:12.272815 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:12.296546 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544994c-edc7-48c3-8de0-21970808da53 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:12.318307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02199e0-603f-4d93-a742-85a3675bc132 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:12.337716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197884cc-7b2d-44a8-b7eb-437d73ebd206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:12.338875 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197884cc-7b2d-44a8-b7eb-437d73ebd206 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:12.339366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197884cc-7b2d-44a8-b7eb-437d73ebd206 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:12.339940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197884cc-7b2d-44a8-b7eb-437d73ebd206 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:12.366776 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b6583-68d1-4912-89d9-e3bd420ace52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:12.368052 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b6583-68d1-4912-89d9-e3bd420ace52 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:12.368672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b6583-68d1-4912-89d9-e3bd420ace52 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:12.369199 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b6583-68d1-4912-89d9-e3bd420ace52 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:12.370214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b6583-68d1-4912-89d9-e3bd420ace52 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:12.370831 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b6583-68d1-4912-89d9-e3bd420ace52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:12.373116 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b6583-68d1-4912-89d9-e3bd420ace52 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:12.379714 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 425/845] 10.222.0.78 () {56 vars in 1114 bytes} [Wed May 25 14:42:12 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:12.465645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197884cc-7b2d-44a8-b7eb-437d73ebd206 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:12.468449 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197884cc-7b2d-44a8-b7eb-437d73ebd206 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:12.576266 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:12.612177 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 421/846] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:42:12 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:42:12.617899 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b6583-68d1-4912-89d9-e3bd420ace52 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:12.740806 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b6583-68d1-4912-89d9-e3bd420ace52 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:12.768618 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:12.780688 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780e003-594f-483c-8d89-075440b2995c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:12.781610 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780e003-594f-483c-8d89-075440b2995c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:12.781897 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780e003-594f-483c-8d89-075440b2995c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:12.782207 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780e003-594f-483c-8d89-075440b2995c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:12.794645 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780e003-594f-483c-8d89-075440b2995c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:12.796245 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780e003-594f-483c-8d89-075440b2995c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:12.816740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-d780e003-594f-483c-8d89-075440b2995c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:12.816740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:12.816740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:12.816740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:12.816740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:12.816740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:12.816740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:12.816740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:12.816740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:12.816740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:12.816740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:12.816740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:12.816740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:12.816740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:12.816740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:12.817697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:12.817697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:12.817697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:12.817697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:12.817697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:12.817697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:12.817697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:12.817697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:12.817697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:12.817697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:12.817697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:12.817697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:12.817697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:12.817697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:12.817697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:12.817697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:12.817697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:12.818623 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:12.818623 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:12.818623 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:12.818623 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:12.818623 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 426/847] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:12.824940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197884cc-7b2d-44a8-b7eb-437d73ebd206 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:12.854545 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362670b2-5601-4cf4-91d7-e5a468176307 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:12.854917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362670b2-5601-4cf4-91d7-e5a468176307 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:12.855216 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362670b2-5601-4cf4-91d7-e5a468176307 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:12.856818 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 427/848] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:42:12.862204 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd715e-937c-42d9-ab33-40e87c6fce3c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:12.862501 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd715e-937c-42d9-ab33-40e87c6fce3c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:12.862731 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd715e-937c-42d9-ab33-40e87c6fce3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:12.882272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-2ddd715e-937c-42d9-ab33-40e87c6fce3c None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:12.883513 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2ddd715e-937c-42d9-ab33-40e87c6fce3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:12.950674 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197884cc-7b2d-44a8-b7eb-437d73ebd206 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:12.990059 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 428/849] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:12.995988 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d6e33-6dec-4f77-984c-4b3d98a27ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:12.996974 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d6e33-6dec-4f77-984c-4b3d98a27ecf None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:12.998067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d6e33-6dec-4f77-984c-4b3d98a27ecf None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:12.998548 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d6e33-6dec-4f77-984c-4b3d98a27ecf None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:12.999611 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d6e33-6dec-4f77-984c-4b3d98a27ecf None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:13.000158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d6e33-6dec-4f77-984c-4b3d98a27ecf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:13.002038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d6e33-6dec-4f77-984c-4b3d98a27ecf None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:13.017872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 422/850] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:13.018147 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd715e-937c-42d9-ab33-40e87c6fce3c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:13.023894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d6e33-6dec-4f77-984c-4b3d98a27ecf None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:13.099626 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:13.179495 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd715e-937c-42d9-ab33-40e87c6fce3c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:13.188680 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:13.190590 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d6e33-6dec-4f77-984c-4b3d98a27ecf None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:13.222218 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f76af-006c-4ab1-b560-e8c635fd7863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:13.223022 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f76af-006c-4ab1-b560-e8c635fd7863 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:13.223310 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f76af-006c-4ab1-b560-e8c635fd7863 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:13.223595 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f76af-006c-4ab1-b560-e8c635fd7863 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:13.232293 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8d604-e3d1-42ee-ac77-eff20fea2a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:13.232963 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8d604-e3d1-42ee-ac77-eff20fea2a5f None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:13.233442 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8d604-e3d1-42ee-ac77-eff20fea2a5f None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:13.233569 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8d604-e3d1-42ee-ac77-eff20fea2a5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:13.236685 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f76af-006c-4ab1-b560-e8c635fd7863 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:42:13.241170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8d604-e3d1-42ee-ac77-eff20fea2a5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:13.242666 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8d604-e3d1-42ee-ac77-eff20fea2a5f None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:13.244473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-4d6f76af-006c-4ab1-b560-e8c635fd7863 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:42:13.244473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:13.244473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:13.244473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:13.244473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:13.244473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:13.244473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:13.244473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:13.244473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:13.244473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:13.244473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:13.244473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:13.244473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:42:13.244473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:42:13.244473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:42:13.244473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:42:13.245599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:42:13.245599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:42:13.245599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:42:13.245599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:42:13.245599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:13.245599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:13.245599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:13.245599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:13.245599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:13.245599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:13.245599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:13.245599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:13.245599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:13.245599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:13.245599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:13.245599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:13.247490 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:13.247490 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:13.247490 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:13.247490 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:13.247490 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:42:13.247490 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:42:13.247490 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:42:13.247490 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:42:13.247490 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:42:13.247490 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:42:13.247490 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:42:13.247490 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:13.247490 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 429/851] 10.222.0.78 () {56 vars in 1102 bytes} [Wed May 25 14:42:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:13.248177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48e91f-8e5f-4958-8bc6-ffdefd3ab154 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:13.248177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48e91f-8e5f-4958-8bc6-ffdefd3ab154 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:13.248177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48e91f-8e5f-4958-8bc6-ffdefd3ab154 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:13.250080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 430/852] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:13.251414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48e91f-8e5f-4958-8bc6-ffdefd3ab154 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:13.264321 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-28b8d604-e3d1-42ee-ac77-eff20fea2a5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:13.264321 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:13.264321 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:13.264321 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:13.264321 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:13.264321 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:13.264321 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:13.264321 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:13.264321 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:13.264321 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:13.264321 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:13.264321 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:13.264321 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:42:13.264321 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:42:13.264321 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:42:13.265279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:42:13.265279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:13.265279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:13.265279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:13.265279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:13.265279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:13.265279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:13.265279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:13.265279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:13.265279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:13.265279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:13.265279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:13.265279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:13.265279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:13.265279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:13.265279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:13.265279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:13.266216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:42:13.266216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:42:13.266216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:42:13.266216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:42:13.266216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:42:13.266216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:42:13.266216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:13.266216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:13.266216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 423/853] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:42:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:13.267113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3e916-77d3-4ed7-9fa1-5965de3f38b5 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:13.267183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3e916-77d3-4ed7-9fa1-5965de3f38b5 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:13.267407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3e916-77d3-4ed7-9fa1-5965de3f38b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:13.289010 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-adb3e916-77d3-4ed7-9fa1-5965de3f38b5 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:13.290240 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-adb3e916-77d3-4ed7-9fa1-5965de3f38b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:13.381766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 424/854] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:13.381766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb3e916-77d3-4ed7-9fa1-5965de3f38b5 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:13.401047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48e91f-8e5f-4958-8bc6-ffdefd3ab154 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:13.428944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:13.459509 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a704a5-bb7d-48f1-8468-13fd20475c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:13.460193 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a704a5-bb7d-48f1-8468-13fd20475c0a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:13.460474 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a704a5-bb7d-48f1-8468-13fd20475c0a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:13.460869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a704a5-bb7d-48f1-8468-13fd20475c0a None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:13.461690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a704a5-bb7d-48f1-8468-13fd20475c0a None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:13.461969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a704a5-bb7d-48f1-8468-13fd20475c0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:13.463366 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a704a5-bb7d-48f1-8468-13fd20475c0a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:13.473536 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 431/855] 10.222.0.78 () {56 vars in 1103 bytes} [Wed May 25 14:42:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:13.478712 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a704a5-bb7d-48f1-8468-13fd20475c0a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:13.509401 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:13.510257 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb3e916-77d3-4ed7-9fa1-5965de3f38b5 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:13.556462 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01e803-bb23-4d3d-8c71-96bf30868050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:13.557221 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01e803-bb23-4d3d-8c71-96bf30868050 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:13.557501 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01e803-bb23-4d3d-8c71-96bf30868050 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:13.557764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01e803-bb23-4d3d-8c71-96bf30868050 None admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:13.558828 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01e803-bb23-4d3d-8c71-96bf30868050 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:13.559137 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01e803-bb23-4d3d-8c71-96bf30868050 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:13.561606 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01e803-bb23-4d3d-8c71-96bf30868050 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:13.587118 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 425/856] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:42:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:13.588678 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01e803-bb23-4d3d-8c71-96bf30868050 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:13.612726 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a704a5-bb7d-48f1-8468-13fd20475c0a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:13.669322 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef3e3b2-580d-4c18-a675-f902622503b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:13.670200 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3e3b2-580d-4c18-a675-f902622503b6 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:13.670501 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3e3b2-580d-4c18-a675-f902622503b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:13.670774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3e3b2-580d-4c18-a675-f902622503b6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:13.671411 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef3e3b2-580d-4c18-a675-f902622503b6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:13.672862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef3e3b2-580d-4c18-a675-f902622503b6 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:13.687181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-3ef3e3b2-580d-4c18-a675-f902622503b6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:42:13.687181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:13.687181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:42:13.687181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:42:13.687181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:42:13.687181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:42:13.687181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:42:13.687181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:13.687181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:42:13.687181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:13.687181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:42:13.687181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:13.687181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:13.687181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:13.687181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:13.687181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:13.688996 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:13.688996 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:13.688996 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:13.688996 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:13.688996 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:13.688996 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:13.688996 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:13.688996 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:42:13.688996 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:42:13.688996 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:42:13.688996 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:42:13.688996 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:13.688996 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:13.688996 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:13.688996 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:13.688996 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:13.690641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:13.690641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:13.690641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:13.690641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:13.690641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:13.690641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:13.690641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:13.690641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:13.690641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:13.690641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:13.690641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:13.690641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:42:13.690641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:13.690641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:42:13.690641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:13.693508 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 432/857] 10.222.0.78 () {56 vars in 1132 bytes} [Wed May 25 14:42:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:13.694663 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef3e3b2-580d-4c18-a675-f902622503b6 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:13.728308 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:13.741059 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01e803-bb23-4d3d-8c71-96bf30868050 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:13.781796 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df5d75-984b-46de-b3b7-c381af78bd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:13.783133 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df5d75-984b-46de-b3b7-c381af78bd55 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:13.783600 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df5d75-984b-46de-b3b7-c381af78bd55 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:13.784093 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df5d75-984b-46de-b3b7-c381af78bd55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:13.801023 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df5d75-984b-46de-b3b7-c381af78bd55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:13.802882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df5d75-984b-46de-b3b7-c381af78bd55 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:13.822638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:13.824066 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-98df5d75-984b-46de-b3b7-c381af78bd55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:13.824066 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:13.824066 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:13.824066 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:13.824066 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:13.824066 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:13.824066 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:13.824066 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:13.824066 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:13.824066 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:13.824066 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:13.824066 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:13.824066 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:13.824066 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:13.824066 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:13.825255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:13.825255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:13.825255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:13.825255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:13.825255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:13.825255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:13.825255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:13.825255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:13.825255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:13.825255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:13.825255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:13.825255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:13.825255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:13.825255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:13.825255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:13.825255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:13.825255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:13.827904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:13.827904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:13.827904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:13.827904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:13.827904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 426/858] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:13.827904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df5d75-984b-46de-b3b7-c381af78bd55 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:13.842870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef3e3b2-580d-4c18-a675-f902622503b6 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:13.891102 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ed07e-d436-41de-90f4-e103b61f5dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:13.892007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ed07e-d436-41de-90f4-e103b61f5dc1 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:13.892372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ed07e-d436-41de-90f4-e103b61f5dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:13.892554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ed07e-d436-41de-90f4-e103b61f5dc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:13.893412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ed07e-d436-41de-90f4-e103b61f5dc1 None admin] RBAC: Adding query filter params (role.id=e566b37cd92f467198b5dc40f183effc, scope.project.id=908a13d2b49544fbaf760b07e213bb7d, user.id=fbe4a2d106d84fc9ac65c046ba675353) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:13.893728 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ed07e-d436-41de-90f4-e103b61f5dc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:13.901906 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ed07e-d436-41de-90f4-e103b61f5dc1 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:13.918299 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 433/859] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:42:13 2022] GET /identity/v3/role_assignments?user.id=fbe4a2d106d84fc9ac65c046ba675353&scope.project.id=908a13d2b49544fbaf760b07e213bb7d&role.id=e566b37cd92f467198b5dc40f183effc => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:13.918299 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ed07e-d436-41de-90f4-e103b61f5dc1 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:13.999256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df5d75-984b-46de-b3b7-c381af78bd55 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:14.053641 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55b088-b142-4a1d-a625-494dde4334a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:14.054277 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55b088-b142-4a1d-a625-494dde4334a6 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:14.054557 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55b088-b142-4a1d-a625-494dde4334a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:14.054831 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55b088-b142-4a1d-a625-494dde4334a6 None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:14.056465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55b088-b142-4a1d-a625-494dde4334a6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:14.056784 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55b088-b142-4a1d-a625-494dde4334a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:14.058421 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55b088-b142-4a1d-a625-494dde4334a6 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:14.070229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 427/860] 10.222.0.78 () {56 vars in 1126 bytes} [Wed May 25 14:42:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:14.071846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55b088-b142-4a1d-a625-494dde4334a6 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:14.088547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:14.135762 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ed07e-d436-41de-90f4-e103b61f5dc1 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:14.202761 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144d0e5-4879-478d-ac4c-97ca6fb0d6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:14.204080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144d0e5-4879-478d-ac4c-97ca6fb0d6a1 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:14.204612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144d0e5-4879-478d-ac4c-97ca6fb0d6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:14.205082 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144d0e5-4879-478d-ac4c-97ca6fb0d6a1 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:14.206370 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144d0e5-4879-478d-ac4c-97ca6fb0d6a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:14.207139 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144d0e5-4879-478d-ac4c-97ca6fb0d6a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:14.209876 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144d0e5-4879-478d-ac4c-97ca6fb0d6a1 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:14.238335 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 434/861] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:14.245837 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144d0e5-4879-478d-ac4c-97ca6fb0d6a1 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:14.253748 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55b088-b142-4a1d-a625-494dde4334a6 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:14.275970 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:14.319133 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0e620-51d0-438c-be24-4ad8fa807709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:14.320249 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0e620-51d0-438c-be24-4ad8fa807709 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:14.320641 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0e620-51d0-438c-be24-4ad8fa807709 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:14.321023 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0e620-51d0-438c-be24-4ad8fa807709 None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:14.322086 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af0e620-51d0-438c-be24-4ad8fa807709 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:14.322524 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af0e620-51d0-438c-be24-4ad8fa807709 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:14.324571 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af0e620-51d0-438c-be24-4ad8fa807709 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:14.332565 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 428/862] 10.222.0.78 () {56 vars in 1126 bytes} [Wed May 25 14:42:14 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:14.405343 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144d0e5-4879-478d-ac4c-97ca6fb0d6a1 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:14.447070 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275661e-9d6f-4667-bd78-22ddd945c208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:14.447962 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275661e-9d6f-4667-bd78-22ddd945c208 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:14.448251 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275661e-9d6f-4667-bd78-22ddd945c208 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:14.448530 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275661e-9d6f-4667-bd78-22ddd945c208 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:14.457876 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6275661e-9d6f-4667-bd78-22ddd945c208 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:14.467689 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6275661e-9d6f-4667-bd78-22ddd945c208 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:14.486359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-6275661e-9d6f-4667-bd78-22ddd945c208 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:42:14.486359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:14.486359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:14.486359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:14.486359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:14.486359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:14.486359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:14.486359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:14.486359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:14.486359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:14.486359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:14.486359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:14.486359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:42:14.486359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:42:14.486359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:42:14.486359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:42:14.488183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:14.488183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:14.488183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:42:14.488183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:14.488183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:42:14.488183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:14.488183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:14.488183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:14.488183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:14.488183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:14.488183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:14.488183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:14.488183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:14.488183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:14.488183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:14.488183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:14.488183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:14.488183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:14.490133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:14.490133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:14.490133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:42:14.490133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:42:14.490133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:42:14.490133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:42:14.490133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:42:14.490133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:42:14.490133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:42:14.490133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:14.490133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 435/863] 10.222.0.78 () {56 vars in 1117 bytes} [Wed May 25 14:42:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:14.495197 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0e620-51d0-438c-be24-4ad8fa807709 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:14.617129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:14.659919 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0e620-51d0-438c-be24-4ad8fa807709 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:14.711547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1cff0-badb-44c7-9fc1-49a2f0d9bb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:14.713198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1cff0-badb-44c7-9fc1-49a2f0d9bb6b None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:14.714057 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1cff0-badb-44c7-9fc1-49a2f0d9bb6b None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:14.714834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1cff0-badb-44c7-9fc1-49a2f0d9bb6b None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:14.717447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e1cff0-badb-44c7-9fc1-49a2f0d9bb6b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:14.718470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e1cff0-badb-44c7-9fc1-49a2f0d9bb6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:14.721495 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e1cff0-badb-44c7-9fc1-49a2f0d9bb6b None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:14.745352 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 429/864] 10.222.0.78 () {56 vars in 1149 bytes} [Wed May 25 14:42:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:14.751055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275661e-9d6f-4667-bd78-22ddd945c208 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:14.855478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:14.881295 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275661e-9d6f-4667-bd78-22ddd945c208 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:14.920014 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d363e0-c5d7-4937-b9a2-c89ffaf3376c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:14.920682 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d363e0-c5d7-4937-b9a2-c89ffaf3376c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:14.921241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d363e0-c5d7-4937-b9a2-c89ffaf3376c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:14.921522 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d363e0-c5d7-4937-b9a2-c89ffaf3376c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:14.922538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563e89d-c989-44b6-a0a6-2dbd3e91499a None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:14.922828 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563e89d-c989-44b6-a0a6-2dbd3e91499a None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:14.923107 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563e89d-c989-44b6-a0a6-2dbd3e91499a None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:14.924865 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 430/865] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:14.930627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b382bd-3cdb-4af6-b052-32fa60fccf5c None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:14.931230 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b382bd-3cdb-4af6-b052-32fa60fccf5c None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:14.931711 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b382bd-3cdb-4af6-b052-32fa60fccf5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:14.954868 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-b8b382bd-3cdb-4af6-b052-32fa60fccf5c None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:14.956586 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-b8b382bd-3cdb-4af6-b052-32fa60fccf5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:14.994902 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d363e0-c5d7-4937-b9a2-c89ffaf3376c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:14.996399 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d363e0-c5d7-4937-b9a2-c89ffaf3376c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:15.050069 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 431/866] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:14 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:42:15.070501 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b382bd-3cdb-4af6-b052-32fa60fccf5c None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:15.078620 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:15.105991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 436/867] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:42:14 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:42:15.111526 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d363e0-c5d7-4937-b9a2-c89ffaf3376c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:15.160663 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:15.219693 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b382bd-3cdb-4af6-b052-32fa60fccf5c None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:15.243140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d363e0-c5d7-4937-b9a2-c89ffaf3376c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:15.271131 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df01ba-65d8-46fe-884b-1ff2e33f1d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:15.272580 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df01ba-65d8-46fe-884b-1ff2e33f1d7e None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:15.273224 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df01ba-65d8-46fe-884b-1ff2e33f1d7e None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:15.273852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df01ba-65d8-46fe-884b-1ff2e33f1d7e None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:15.275413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df01ba-65d8-46fe-884b-1ff2e33f1d7e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:15.278025 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df01ba-65d8-46fe-884b-1ff2e33f1d7e None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:15.302844 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d19b7-8efc-401c-939e-fbf52b272bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:15.304281 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d19b7-8efc-401c-939e-fbf52b272bd5 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:15.304856 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d19b7-8efc-401c-939e-fbf52b272bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:15.305300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d19b7-8efc-401c-939e-fbf52b272bd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:15.312542 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 432/868] 10.222.0.78 () {60 vars in 1141 bytes} [Wed May 25 14:42:15 2022] POST /identity/v3/services => generated 251 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:42:15.328085 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004d19b7-8efc-401c-939e-fbf52b272bd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:15.329868 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004d19b7-8efc-401c-939e-fbf52b272bd5 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:15.363836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-004d19b7-8efc-401c-939e-fbf52b272bd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:15.363836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:15.363836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:15.363836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:15.363836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:15.363836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:15.363836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:15.363836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:15.363836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:15.363836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:15.363836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:15.363836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:15.363836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:15.363836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:15.363836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:15.365438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:15.365438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:15.365438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:15.365438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:15.365438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:15.365438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:15.365438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:15.365438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:15.365438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:15.365438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:15.365438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:15.365438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:15.365438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:15.365438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:15.365438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:15.365438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:15.365438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:15.367377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:15.367377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:15.367377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:15.367377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:15.367377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 437/869] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:15.372609 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df01ba-65d8-46fe-884b-1ff2e33f1d7e None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:15.588564 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df01ba-65d8-46fe-884b-1ff2e33f1d7e None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:15.621404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:15.654130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada6f3f-db12-4578-9664-4a37d11edf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:15.655107 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada6f3f-db12-4578-9664-4a37d11edf5c None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:15.655494 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada6f3f-db12-4578-9664-4a37d11edf5c None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:15.655868 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada6f3f-db12-4578-9664-4a37d11edf5c None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:15.656919 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ada6f3f-db12-4578-9664-4a37d11edf5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:15.657350 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ada6f3f-db12-4578-9664-4a37d11edf5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:15.659392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ada6f3f-db12-4578-9664-4a37d11edf5c None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:15.678471 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 433/870] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:15.686677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d19b7-8efc-401c-939e-fbf52b272bd5 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:15.728512 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:15.857442 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d19b7-8efc-401c-939e-fbf52b272bd5 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:15.923604 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a92a00-a306-4c7a-8e5e-c5b1ee5b3d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:15.924961 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a92a00-a306-4c7a-8e5e-c5b1ee5b3d74 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:15.925443 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a92a00-a306-4c7a-8e5e-c5b1ee5b3d74 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:15.925923 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a92a00-a306-4c7a-8e5e-c5b1ee5b3d74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:15.938889 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a92a00-a306-4c7a-8e5e-c5b1ee5b3d74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:15.941604 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a92a00-a306-4c7a-8e5e-c5b1ee5b3d74 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:15.968558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-f0a92a00-a306-4c7a-8e5e-c5b1ee5b3d74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:15.968558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:15.968558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:15.968558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:15.968558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:15.968558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:15.968558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:15.968558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:15.968558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:15.968558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:15.968558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:15.968558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:15.968558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:42:15.968558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:42:15.968558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:42:15.969531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:42:15.969531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:15.969531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:15.969531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:15.969531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:15.969531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:15.969531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:15.969531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:15.969531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:15.969531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:15.969531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:15.969531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:15.969531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:15.969531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:15.969531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:15.969531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:15.969531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:15.970577 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:42:15.970577 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:42:15.970577 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:42:15.970577 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:42:15.970577 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:42:15.970577 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:42:15.970577 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:15.970577 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:15.971466 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 438/871] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:42:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:15.978511 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada6f3f-db12-4578-9664-4a37d11edf5c None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:16.104424 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada6f3f-db12-4578-9664-4a37d11edf5c None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:16.144519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bef112-8302-4bcd-a15e-b29daa521628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:16.145951 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bef112-8302-4bcd-a15e-b29daa521628 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:16.146703 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bef112-8302-4bcd-a15e-b29daa521628 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:16.147388 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bef112-8302-4bcd-a15e-b29daa521628 None admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:16.148967 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bef112-8302-4bcd-a15e-b29daa521628 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:16.149712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bef112-8302-4bcd-a15e-b29daa521628 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:16.152227 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bef112-8302-4bcd-a15e-b29daa521628 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:16.174121 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 434/872] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:42:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:16.191756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a92a00-a306-4c7a-8e5e-c5b1ee5b3d74 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:16.391169 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:16.415156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a92a00-a306-4c7a-8e5e-c5b1ee5b3d74 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:16.485364 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0072ba5-3c88-4bb2-992a-8ab8c994510b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:16.486549 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0072ba5-3c88-4bb2-992a-8ab8c994510b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:16.487065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0072ba5-3c88-4bb2-992a-8ab8c994510b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:16.487569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0072ba5-3c88-4bb2-992a-8ab8c994510b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:16.489333 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0072ba5-3c88-4bb2-992a-8ab8c994510b None admin] RBAC: Adding query filter params (role.id=336f27e7f8624a30b30cef9acada97a0, scope.project.id=908a13d2b49544fbaf760b07e213bb7d, user.id=b2935579c8554bed8b1e211c47548ac5) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:16.489909 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0072ba5-3c88-4bb2-992a-8ab8c994510b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:16.492744 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0072ba5-3c88-4bb2-992a-8ab8c994510b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:16.505491 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 439/873] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:42:16 2022] GET /identity/v3/role_assignments?user.id=b2935579c8554bed8b1e211c47548ac5&scope.project.id=908a13d2b49544fbaf760b07e213bb7d&role.id=336f27e7f8624a30b30cef9acada97a0 => generated 608 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:16.653888 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433c5c0-c769-4ebf-ab3a-4aaac088263c None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:16.654724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433c5c0-c769-4ebf-ab3a-4aaac088263c None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:16.654724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433c5c0-c769-4ebf-ab3a-4aaac088263c None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:16.656336 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 435/874] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:16.662687 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b213f0-308a-45e7-98ba-314e05b4dd93 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:16.663108 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b213f0-308a-45e7-98ba-314e05b4dd93 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:16.663251 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b213f0-308a-45e7-98ba-314e05b4dd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:16.682829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-d9b213f0-308a-45e7-98ba-314e05b4dd93 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:16.684027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d9b213f0-308a-45e7-98ba-314e05b4dd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:16.781256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 440/875] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:16 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:16.821967 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433c5c0-c769-4ebf-ab3a-4aaac088263c None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:16.897815 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:17.026274 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433c5c0-c769-4ebf-ab3a-4aaac088263c None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:17.095922 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4f0eb-f4c1-4e3a-bfbd-c2ddaba2aeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:17.097817 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4f0eb-f4c1-4e3a-bfbd-c2ddaba2aeb0 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:17.098626 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4f0eb-f4c1-4e3a-bfbd-c2ddaba2aeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:17.099404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4f0eb-f4c1-4e3a-bfbd-c2ddaba2aeb0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:17.100938 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4f0eb-f4c1-4e3a-bfbd-c2ddaba2aeb0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:17.103856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4f0eb-f4c1-4e3a-bfbd-c2ddaba2aeb0 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:17.128696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-74e4f0eb-f4c1-4e3a-bfbd-c2ddaba2aeb0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:42:17.128696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:17.128696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:42:17.128696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:42:17.128696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:42:17.128696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:42:17.128696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:42:17.128696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:17.128696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:42:17.128696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:17.128696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:42:17.128696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:17.128696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:17.128696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:17.128696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:17.128696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:17.128696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:17.130647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:17.130647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:17.130647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:17.130647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:17.130647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:17.130647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:17.130647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:42:17.130647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:42:17.130647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:42:17.130647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:42:17.130647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:17.130647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:17.130647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:17.130647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:17.130647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:17.130647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:17.132264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:17.132264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:17.132264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:17.132264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:17.132264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:17.132264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:17.132264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:17.132264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:17.132264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:17.132264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:17.132264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:42:17.132264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:17.132264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:42:17.132264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:17.137166 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 436/876] 10.222.0.78 () {56 vars in 1126 bytes} [Wed May 25 14:42:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:17.148358 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b213f0-308a-45e7-98ba-314e05b4dd93 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:17.177587 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55578bf5-545b-47f5-a100-4b2226a3d0a8 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:17.177872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55578bf5-545b-47f5-a100-4b2226a3d0a8 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:17.178209 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55578bf5-545b-47f5-a100-4b2226a3d0a8 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:17.179849 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 437/877] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:42:17.185435 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487bbf7-e7a0-4043-8ee8-f10d57131544 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:17.185764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487bbf7-e7a0-4043-8ee8-f10d57131544 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:17.186000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487bbf7-e7a0-4043-8ee8-f10d57131544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:17.205562 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-f487bbf7-e7a0-4043-8ee8-f10d57131544 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:17.206814 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f487bbf7-e7a0-4043-8ee8-f10d57131544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:17.305340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 438/878] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:17 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:42:17.330902 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487bbf7-e7a0-4043-8ee8-f10d57131544 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:17.346385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:17.396357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b213f0-308a-45e7-98ba-314e05b4dd93 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:17.399590 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:17.467175 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57654b1-6ef1-4e36-b586-49e0d58d789c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:17.468701 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57654b1-6ef1-4e36-b586-49e0d58d789c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:17.469204 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57654b1-6ef1-4e36-b586-49e0d58d789c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:17.469689 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57654b1-6ef1-4e36-b586-49e0d58d789c None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:17.471067 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57654b1-6ef1-4e36-b586-49e0d58d789c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:17.472216 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57654b1-6ef1-4e36-b586-49e0d58d789c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:17.475645 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57654b1-6ef1-4e36-b586-49e0d58d789c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:17.486258 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 441/879] 10.222.0.78 () {56 vars in 1122 bytes} [Wed May 25 14:42:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:17.494910 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57654b1-6ef1-4e36-b586-49e0d58d789c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:17.550937 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487bbf7-e7a0-4043-8ee8-f10d57131544 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:17.596966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b4886-0c26-4cfc-80d5-46e776aff636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:17.598976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b4886-0c26-4cfc-80d5-46e776aff636 None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:17.599848 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b4886-0c26-4cfc-80d5-46e776aff636 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:17.600649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b4886-0c26-4cfc-80d5-46e776aff636 None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:17.602577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b4886-0c26-4cfc-80d5-46e776aff636 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:17.605156 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b4886-0c26-4cfc-80d5-46e776aff636 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:17.646634 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 439/880] 10.222.0.78 () {60 vars in 1141 bytes} [Wed May 25 14:42:17 2022] POST /identity/v3/services => generated 269 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:42:17.651229 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57654b1-6ef1-4e36-b586-49e0d58d789c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:17.708979 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc768d1f-29b4-4e94-9a39-35a438ea4d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:17.709663 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc768d1f-29b4-4e94-9a39-35a438ea4d35 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:17.709938 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc768d1f-29b4-4e94-9a39-35a438ea4d35 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:17.710240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc768d1f-29b4-4e94-9a39-35a438ea4d35 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:17.710964 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc768d1f-29b4-4e94-9a39-35a438ea4d35 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:17.711266 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc768d1f-29b4-4e94-9a39-35a438ea4d35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:17.712686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc768d1f-29b4-4e94-9a39-35a438ea4d35 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:17.719016 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 442/881] 10.222.0.78 () {56 vars in 1122 bytes} [Wed May 25 14:42:17 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:18.209048 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3ada6-f65a-47af-b85b-dbb723e61606 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:18.210089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3ada6-f65a-47af-b85b-dbb723e61606 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:18.210633 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3ada6-f65a-47af-b85b-dbb723e61606 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:18.212518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 440/882] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:18.219000 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08265a6e-5c41-4a4f-afa5-0bb5ffd93580 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:18.219905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08265a6e-5c41-4a4f-afa5-0bb5ffd93580 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:18.220394 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08265a6e-5c41-4a4f-afa5-0bb5ffd93580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:18.232710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:18.240829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-08265a6e-5c41-4a4f-afa5-0bb5ffd93580 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:18.242460 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-08265a6e-5c41-4a4f-afa5-0bb5ffd93580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:18.329720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 443/883] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:18 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:18.353145 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3ada6-f65a-47af-b85b-dbb723e61606 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:18.462118 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:18.476727 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3ada6-f65a-47af-b85b-dbb723e61606 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:18.516675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1691a1e-7043-43b1-8db3-de41b2120228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:18.518175 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1691a1e-7043-43b1-8db3-de41b2120228 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:18.518708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1691a1e-7043-43b1-8db3-de41b2120228 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:18.519182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1691a1e-7043-43b1-8db3-de41b2120228 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:18.520234 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1691a1e-7043-43b1-8db3-de41b2120228 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:18.521960 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1691a1e-7043-43b1-8db3-de41b2120228 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:18.536680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-c1691a1e-7043-43b1-8db3-de41b2120228 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:42:18.536680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:18.536680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:42:18.536680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:42:18.536680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:42:18.536680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:42:18.536680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:42:18.536680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:18.536680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:42:18.536680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:18.536680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:42:18.536680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:18.536680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:18.536680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:18.536680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:18.536680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:18.536680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:18.537730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:18.537730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:18.537730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:18.537730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:18.537730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:18.537730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:18.537730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:42:18.537730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:42:18.537730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:42:18.537730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:42:18.537730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:18.537730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:18.537730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:18.537730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:18.537730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:18.537730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:18.538617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:18.538617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:18.538617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:18.538617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:18.538617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:18.538617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:18.538617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:18.538617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:18.538617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:18.538617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:18.538617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:42:18.538617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:18.538617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:42:18.538617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:18.541877 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 441/884] 10.222.0.78 () {56 vars in 1114 bytes} [Wed May 25 14:42:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:18.551244 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08265a6e-5c41-4a4f-afa5-0bb5ffd93580 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:18.674719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08265a6e-5c41-4a4f-afa5-0bb5ffd93580 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:18.715444 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb32ab-f634-4c01-9de7-695190229244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:18.717057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb32ab-f634-4c01-9de7-695190229244 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:18.717548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb32ab-f634-4c01-9de7-695190229244 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:18.718072 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb32ab-f634-4c01-9de7-695190229244 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:18.718998 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb32ab-f634-4c01-9de7-695190229244 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:18.719530 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb32ab-f634-4c01-9de7-695190229244 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:18.721148 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb32ab-f634-4c01-9de7-695190229244 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:18.723671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:18.728091 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 444/885] 10.222.0.78 () {56 vars in 1114 bytes} [Wed May 25 14:42:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:18.734351 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1691a1e-7043-43b1-8db3-de41b2120228 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:18.854170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1691a1e-7043-43b1-8db3-de41b2120228 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:18.895107 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc2598-2925-4595-88c6-c63abce76efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:18.896151 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc2598-2925-4595-88c6-c63abce76efb None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:18.896653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc2598-2925-4595-88c6-c63abce76efb None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:18.897185 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc2598-2925-4595-88c6-c63abce76efb None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:18.898154 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc2598-2925-4595-88c6-c63abce76efb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:18.898693 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc2598-2925-4595-88c6-c63abce76efb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:18.900392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc2598-2925-4595-88c6-c63abce76efb None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:18.912517 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 442/886] 10.222.0.78 () {56 vars in 1114 bytes} [Wed May 25 14:42:18 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:18.913908 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb32ab-f634-4c01-9de7-695190229244 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:19.070127 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb32ab-f634-4c01-9de7-695190229244 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:19.112233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813e70f-d7c8-4646-aca7-947601173612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:19.113353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813e70f-d7c8-4646-aca7-947601173612 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:19.113936 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813e70f-d7c8-4646-aca7-947601173612 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:19.114451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813e70f-d7c8-4646-aca7-947601173612 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:19.115471 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e813e70f-d7c8-4646-aca7-947601173612 None admin] RBAC: Adding query filter params (interface=public, service_id=bcef35247d4041bead4f371b929253ae, region_id=RegionOne) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:19.115995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e813e70f-d7c8-4646-aca7-947601173612 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:19.117594 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e813e70f-d7c8-4646-aca7-947601173612 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:19.124259 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 445/887] 10.222.0.78 () {56 vars in 1251 bytes} [Wed May 25 14:42:18 2022] GET /identity/v3/endpoints?service_id=bcef35247d4041bead4f371b929253ae&interface=public®ion_id=RegionOne => generated 194 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:19.130843 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc2598-2925-4595-88c6-c63abce76efb None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:19.167773 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:19.279518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc2598-2925-4595-88c6-c63abce76efb None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:19.295268 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51fac4-3dde-45df-8bff-879fac981fb3 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:19.295830 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51fac4-3dde-45df-8bff-879fac981fb3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:19.296232 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51fac4-3dde-45df-8bff-879fac981fb3 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:19.305313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 446/888] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:19.308265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a9994-1551-460b-8653-936d5c39f56c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:19.308777 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a9994-1551-460b-8653-936d5c39f56c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:19.309286 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a9994-1551-460b-8653-936d5c39f56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:19.324185 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e84e0a-c907-43a6-b73c-0d4870477858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:19.325243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e84e0a-c907-43a6-b73c-0d4870477858 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:19.325746 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e84e0a-c907-43a6-b73c-0d4870477858 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:19.326254 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e84e0a-c907-43a6-b73c-0d4870477858 None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:19.327160 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e84e0a-c907-43a6-b73c-0d4870477858 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:19.328750 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e84e0a-c907-43a6-b73c-0d4870477858 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:19.336580 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 443/889] 10.222.0.78 () {56 vars in 1087 bytes} [Wed May 25 14:42:19 2022] GET /identity/v3/services => generated 1268 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:42:19.342046 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-db5a9994-1551-460b-8653-936d5c39f56c None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:19.342046 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-db5a9994-1551-460b-8653-936d5c39f56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:19.466500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:19.490200 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 447/890] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:19 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:42:19.529709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e84e0a-c907-43a6-b73c-0d4870477858 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:19.667450 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:19.690960 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e84e0a-c907-43a6-b73c-0d4870477858 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:19.734798 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af72f0-620e-40e6-be8f-4dfbb676463b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:19.735627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af72f0-620e-40e6-be8f-4dfbb676463b None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:19.735907 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af72f0-620e-40e6-be8f-4dfbb676463b None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:19.736180 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af72f0-620e-40e6-be8f-4dfbb676463b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:19.749177 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af72f0-620e-40e6-be8f-4dfbb676463b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:42:19.757622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-a3af72f0-620e-40e6-be8f-4dfbb676463b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:42:19.757622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:19.757622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:19.757622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:19.757622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:19.757622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:19.757622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:19.757622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:19.757622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:19.757622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:19.757622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:19.757622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:19.757622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:42:19.757622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:42:19.757622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:42:19.757622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:42:19.758722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:42:19.758722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:42:19.758722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:42:19.758722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:42:19.758722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:19.758722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:19.758722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:19.758722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:19.758722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:19.758722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:19.758722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:19.758722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:19.758722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:19.758722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:19.758722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:19.758722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:19.759649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:19.759649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:19.759649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:19.759649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:19.759649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:42:19.759649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:42:19.759649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:42:19.759649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:42:19.759649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:42:19.759649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:42:19.759649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:42:19.759649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:19.759649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 444/891] 10.222.0.78 () {56 vars in 1120 bytes} [Wed May 25 14:42:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:19.765953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a9994-1551-460b-8653-936d5c39f56c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:19.919399 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a9994-1551-460b-8653-936d5c39f56c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:19.959199 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6efdc-d396-4939-a7cb-8be4fd3251ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:19.959898 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6efdc-d396-4939-a7cb-8be4fd3251ad None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:19.960182 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6efdc-d396-4939-a7cb-8be4fd3251ad None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:19.960455 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6efdc-d396-4939-a7cb-8be4fd3251ad None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:19.961229 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6efdc-d396-4939-a7cb-8be4fd3251ad None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:19.961532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6efdc-d396-4939-a7cb-8be4fd3251ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:19.963024 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6efdc-d396-4939-a7cb-8be4fd3251ad None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:19.975342 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 448/892] 10.222.0.78 () {56 vars in 1115 bytes} [Wed May 25 14:42:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:19.985787 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af72f0-620e-40e6-be8f-4dfbb676463b None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:20.117837 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af72f0-620e-40e6-be8f-4dfbb676463b None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:20.160160 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9b430-e8b6-421b-86f5-1fec9cb8e010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:20.161272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9b430-e8b6-421b-86f5-1fec9cb8e010 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:20.161719 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9b430-e8b6-421b-86f5-1fec9cb8e010 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:20.162196 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9b430-e8b6-421b-86f5-1fec9cb8e010 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:20.175988 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9b430-e8b6-421b-86f5-1fec9cb8e010 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:20.177721 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9b430-e8b6-421b-86f5-1fec9cb8e010 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:20.180034 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:20.199361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-0ee9b430-e8b6-421b-86f5-1fec9cb8e010 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:20.199361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:20.199361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:20.199361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:20.199361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:20.199361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:20.199361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:20.199361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:20.199361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:20.199361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:20.199361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:20.199361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:20.199361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:20.199361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:20.199361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:20.201026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:20.201026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:20.201026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:20.201026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:20.201026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:20.201026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:20.201026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:20.201026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:20.201026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:20.201026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:20.201026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:20.201026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:20.201026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:20.201026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:20.201026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:20.201026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:20.201026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:20.201943 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:20.201943 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:20.201943 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:20.201943 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:20.201943 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 445/893] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:20.210773 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6efdc-d396-4939-a7cb-8be4fd3251ad None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:20.259344 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862aa215-e422-429b-b757-89c9e9c963fc None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:20.259646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862aa215-e422-429b-b757-89c9e9c963fc None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:20.260033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862aa215-e422-429b-b757-89c9e9c963fc None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:20.261836 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 446/894] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:20.268458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d71b6b-1972-419f-a230-3051c02c4c1d None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:20.268733 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d71b6b-1972-419f-a230-3051c02c4c1d None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:20.269044 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d71b6b-1972-419f-a230-3051c02c4c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:20.299816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-42d71b6b-1972-419f-a230-3051c02c4c1d None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:20.303332 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-42d71b6b-1972-419f-a230-3051c02c4c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:20.308583 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:20.367681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6efdc-d396-4939-a7cb-8be4fd3251ad None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:20.412629 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24ccd7-5ea5-477f-bc41-63b8d8c57949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:20.413322 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24ccd7-5ea5-477f-bc41-63b8d8c57949 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:20.413645 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24ccd7-5ea5-477f-bc41-63b8d8c57949 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:20.413922 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24ccd7-5ea5-477f-bc41-63b8d8c57949 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:20.414729 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a24ccd7-5ea5-477f-bc41-63b8d8c57949 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:20.415032 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a24ccd7-5ea5-477f-bc41-63b8d8c57949 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:20.416435 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a24ccd7-5ea5-477f-bc41-63b8d8c57949 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:20.434664 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 449/895] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:20.443241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24ccd7-5ea5-477f-bc41-63b8d8c57949 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:20.455353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 447/896] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:42:20.503792 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d71b6b-1972-419f-a230-3051c02c4c1d None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:20.571265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24ccd7-5ea5-477f-bc41-63b8d8c57949 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:20.594148 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:20.619568 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948cd87-3999-46de-92d4-1f9ec2224cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:20.620407 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948cd87-3999-46de-92d4-1f9ec2224cfa None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:20.620687 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948cd87-3999-46de-92d4-1f9ec2224cfa None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:20.620950 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948cd87-3999-46de-92d4-1f9ec2224cfa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:20.630589 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6948cd87-3999-46de-92d4-1f9ec2224cfa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:20.632083 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6948cd87-3999-46de-92d4-1f9ec2224cfa None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:20.651507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-6948cd87-3999-46de-92d4-1f9ec2224cfa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:42:20.651507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:20.651507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:20.651507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:20.651507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:20.651507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:20.651507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:20.651507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:20.651507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:20.651507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:20.651507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:20.651507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:20.651507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:42:20.651507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:42:20.651507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:42:20.651507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:42:20.662414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:20.662414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:20.662414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:42:20.662414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:20.662414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:42:20.662414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:20.662414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:20.662414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:20.662414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:20.662414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:20.662414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:20.662414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:20.662414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:20.662414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:20.662414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:20.662414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:20.662414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:20.662414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:20.664349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:20.664349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:20.664349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:42:20.664349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:42:20.664349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:42:20.664349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:42:20.664349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:42:20.664349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:42:20.664349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:42:20.664349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:20.664349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 450/897] 10.222.0.78 () {56 vars in 1117 bytes} [Wed May 25 14:42:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:20.669434 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948cd87-3999-46de-92d4-1f9ec2224cfa None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:20.686256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d71b6b-1972-419f-a230-3051c02c4c1d None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:20.717026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:20.734371 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18224be3-136d-4573-8b9c-a34d4a157523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:20.735049 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18224be3-136d-4573-8b9c-a34d4a157523 None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:20.735326 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18224be3-136d-4573-8b9c-a34d4a157523 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:20.735611 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18224be3-136d-4573-8b9c-a34d4a157523 None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:20.736335 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18224be3-136d-4573-8b9c-a34d4a157523 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:20.738052 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18224be3-136d-4573-8b9c-a34d4a157523 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:20.762048 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 448/898] 10.222.0.78 () {60 vars in 1141 bytes} [Wed May 25 14:42:20 2022] POST /identity/v3/services => generated 241 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:42:20.834156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948cd87-3999-46de-92d4-1f9ec2224cfa None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:20.885243 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f870f6-59f2-49f3-9568-4f83e68ddc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:20.885935 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f870f6-59f2-49f3-9568-4f83e68ddc60 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:20.886251 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f870f6-59f2-49f3-9568-4f83e68ddc60 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:20.886527 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f870f6-59f2-49f3-9568-4f83e68ddc60 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:20.887930 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f870f6-59f2-49f3-9568-4f83e68ddc60 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:20.888240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f870f6-59f2-49f3-9568-4f83e68ddc60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:20.889761 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f870f6-59f2-49f3-9568-4f83e68ddc60 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:20.912250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 451/899] 10.222.0.78 () {56 vars in 1149 bytes} [Wed May 25 14:42:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:20.923182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18224be3-136d-4573-8b9c-a34d4a157523 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:21.077234 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18224be3-136d-4573-8b9c-a34d4a157523 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:21.145471 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482f70d-ceca-416f-b46c-8218e09a1df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:21.147204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482f70d-ceca-416f-b46c-8218e09a1df6 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:21.148105 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482f70d-ceca-416f-b46c-8218e09a1df6 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:21.148888 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482f70d-ceca-416f-b46c-8218e09a1df6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:21.183461 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729d4c2-bb87-445f-b0ef-66835e156569 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:21.184804 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729d4c2-bb87-445f-b0ef-66835e156569 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:21.185779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729d4c2-bb87-445f-b0ef-66835e156569 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:21.190064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 452/900] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:21.200012 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc5a0b-051d-468d-9736-444baff420e6 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:21.200516 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc5a0b-051d-468d-9736-444baff420e6 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:21.201005 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc5a0b-051d-468d-9736-444baff420e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:21.229246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-f4dc5a0b-051d-468d-9736-444baff420e6 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:21.231921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f4dc5a0b-051d-468d-9736-444baff420e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:21.265384 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:21.269674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:21.286616 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c482f70d-ceca-416f-b46c-8218e09a1df6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:21.288116 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c482f70d-ceca-416f-b46c-8218e09a1df6 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:21.361096 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 453/901] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:42:21.384684 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc5a0b-051d-468d-9736-444baff420e6 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:21.425160 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 449/902] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:42:20 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:42:21.435365 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482f70d-ceca-416f-b46c-8218e09a1df6 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:21.602043 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:21.611400 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc5a0b-051d-468d-9736-444baff420e6 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:21.641332 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482f70d-ceca-416f-b46c-8218e09a1df6 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:21.680682 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1526a-4d2b-4311-9ff8-e7dc353904a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:21.681958 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1526a-4d2b-4311-9ff8-e7dc353904a9 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:21.682438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1526a-4d2b-4311-9ff8-e7dc353904a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:21.682903 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1526a-4d2b-4311-9ff8-e7dc353904a9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:21.684793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1526a-4d2b-4311-9ff8-e7dc353904a9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:21.687215 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1526a-4d2b-4311-9ff8-e7dc353904a9 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:21.710987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-b2a1526a-4d2b-4311-9ff8-e7dc353904a9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:42:21.710987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:21.710987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:42:21.710987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:42:21.710987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:42:21.710987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:42:21.710987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:42:21.710987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:21.710987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:42:21.710987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:21.710987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:42:21.710987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:21.710987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:21.710987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:21.710987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:21.710987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:21.712969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:21.712969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:21.712969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:21.712969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:21.712969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:21.712969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:21.712969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:21.712969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:42:21.712969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:42:21.712969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:42:21.712969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:42:21.712969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:21.712969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:21.712969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:21.712969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:21.712969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:21.716225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:21.716225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:21.716225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:21.716225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:21.716225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:21.716225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:21.716225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:21.716225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:21.716225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:21.716225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:21.716225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:21.716225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:42:21.716225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:21.716225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:42:21.716225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:21.716225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 454/903] 10.222.0.78 () {56 vars in 1132 bytes} [Wed May 25 14:42:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:21.720935 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1526a-4d2b-4311-9ff8-e7dc353904a9 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:21.731217 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f933f-33bd-4391-a2ae-fad7f2a9e1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:21.732441 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f933f-33bd-4391-a2ae-fad7f2a9e1df None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:21.732857 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f933f-33bd-4391-a2ae-fad7f2a9e1df None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:21.733255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f933f-33bd-4391-a2ae-fad7f2a9e1df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:21.758833 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f933f-33bd-4391-a2ae-fad7f2a9e1df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:21.763032 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f933f-33bd-4391-a2ae-fad7f2a9e1df None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:21.785066 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:21.810140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-782f933f-33bd-4391-a2ae-fad7f2a9e1df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:21.810140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:21.810140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:21.810140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:21.810140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:21.810140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:21.810140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:21.810140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:21.810140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:21.810140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:21.810140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:21.810140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:21.810140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:21.810140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:21.810140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:21.811949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:21.811949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:21.811949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:21.811949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:21.811949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:21.811949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:21.811949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:21.811949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:21.811949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:21.811949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:21.811949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:21.811949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:21.811949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:21.811949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:21.811949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:21.811949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:21.811949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:21.813756 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:21.813756 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:21.813756 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:21.813756 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:21.818169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 450/904] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:21.827848 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f933f-33bd-4391-a2ae-fad7f2a9e1df None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:21.947117 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1526a-4d2b-4311-9ff8-e7dc353904a9 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:21.991422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f933f-33bd-4391-a2ae-fad7f2a9e1df None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:21.993921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17369d6c-00aa-482b-8045-5a187c65405d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:21.995057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17369d6c-00aa-482b-8045-5a187c65405d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:21.995057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17369d6c-00aa-482b-8045-5a187c65405d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:21.995234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17369d6c-00aa-482b-8045-5a187c65405d None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:21.995893 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17369d6c-00aa-482b-8045-5a187c65405d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:21.996209 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17369d6c-00aa-482b-8045-5a187c65405d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:21.998209 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17369d6c-00aa-482b-8045-5a187c65405d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:22.003927 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 455/905] 10.222.0.78 () {56 vars in 1126 bytes} [Wed May 25 14:42:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:22.005308 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b028c2c-dab8-45e7-a99b-f2b06183b280 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:22.005598 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b028c2c-dab8-45e7-a99b-f2b06183b280 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:22.005882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b028c2c-dab8-45e7-a99b-f2b06183b280 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:22.011397 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 456/906] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:22.013049 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b028c2c-dab8-45e7-a99b-f2b06183b280 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:22.065065 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc848c8-5374-4db9-b855-d611f1ae6b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:22.065745 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc848c8-5374-4db9-b855-d611f1ae6b5a None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:22.066088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc848c8-5374-4db9-b855-d611f1ae6b5a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:22.066342 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc848c8-5374-4db9-b855-d611f1ae6b5a None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:22.067232 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc848c8-5374-4db9-b855-d611f1ae6b5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:22.067411 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc848c8-5374-4db9-b855-d611f1ae6b5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:22.068754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc848c8-5374-4db9-b855-d611f1ae6b5a None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:22.071733 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:22.091478 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 451/907] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:22.091478 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a97b6e-db8a-4a83-b60e-40fb2f409e2e None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:22.091478 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a97b6e-db8a-4a83-b60e-40fb2f409e2e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:22.091478 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a97b6e-db8a-4a83-b60e-40fb2f409e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:22.104794 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-f7a97b6e-db8a-4a83-b60e-40fb2f409e2e None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:22.106128 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f7a97b6e-db8a-4a83-b60e-40fb2f409e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:22.156835 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:22.184250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b028c2c-dab8-45e7-a99b-f2b06183b280 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:22.202973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 452/908] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:22.204409 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a97b6e-db8a-4a83-b60e-40fb2f409e2e None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:22.246084 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c01ee-f017-4194-9c46-9e02eb60cb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:22.246953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c01ee-f017-4194-9c46-9e02eb60cb49 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:22.247260 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c01ee-f017-4194-9c46-9e02eb60cb49 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:22.247642 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c01ee-f017-4194-9c46-9e02eb60cb49 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:22.248452 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c01ee-f017-4194-9c46-9e02eb60cb49 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:22.248849 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c01ee-f017-4194-9c46-9e02eb60cb49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:22.250425 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c01ee-f017-4194-9c46-9e02eb60cb49 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:22.260914 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 457/909] 10.222.0.78 () {56 vars in 1126 bytes} [Wed May 25 14:42:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:22.260914 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c01ee-f017-4194-9c46-9e02eb60cb49 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:22.360061 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a97b6e-db8a-4a83-b60e-40fb2f409e2e None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:22.391742 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:22.406294 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb0f1f-d762-4516-8380-4892500c6ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:22.407639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb0f1f-d762-4516-8380-4892500c6ba8 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:22.407832 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb0f1f-d762-4516-8380-4892500c6ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:22.408120 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb0f1f-d762-4516-8380-4892500c6ba8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:22.417493 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c01ee-f017-4194-9c46-9e02eb60cb49 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:22.421123 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb0f1f-d762-4516-8380-4892500c6ba8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:22.423686 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb0f1f-d762-4516-8380-4892500c6ba8 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:22.439991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-41cb0f1f-d762-4516-8380-4892500c6ba8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:22.439991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:22.439991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:22.439991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:22.439991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:22.439991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:22.439991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:22.439991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:22.439991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:22.439991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:22.439991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:22.439991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:22.439991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:42:22.439991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:42:22.439991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:42:22.442856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:42:22.442856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:22.442856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:22.442856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:22.442856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:22.442856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:22.442856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:22.442856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:22.442856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:22.442856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:22.442856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:22.442856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:22.442856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:22.442856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:22.442856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:22.442856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:22.442856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:22.444338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:42:22.444338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:42:22.444338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:42:22.444338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:42:22.444338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:42:22.444338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:42:22.444338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:22.444338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:22.444338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 453/910] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:42:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:22.444338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb0f1f-d762-4516-8380-4892500c6ba8 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:22.495172 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d53cd5-8bb9-401a-b3af-ccc4f84cbe53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:22.497708 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d53cd5-8bb9-401a-b3af-ccc4f84cbe53 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:22.497708 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d53cd5-8bb9-401a-b3af-ccc4f84cbe53 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:22.498207 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d53cd5-8bb9-401a-b3af-ccc4f84cbe53 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:22.499388 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d53cd5-8bb9-401a-b3af-ccc4f84cbe53 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:22.501992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d53cd5-8bb9-401a-b3af-ccc4f84cbe53 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:22.526029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-55d53cd5-8bb9-401a-b3af-ccc4f84cbe53 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:42:22.526029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:22.526029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:42:22.526029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:42:22.526029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:42:22.526029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:42:22.526029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:42:22.526029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:22.526029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:42:22.526029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:22.526029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:42:22.526029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:22.526029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:22.526029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:22.526029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:22.526029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:22.526029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:22.528575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:22.528575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:22.528575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:22.528575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:22.528575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:22.528575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:22.528575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:42:22.528575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:42:22.528575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:42:22.528575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:42:22.528575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:22.528575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:22.528575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:22.528575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:22.528575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:22.528575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:22.529995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:22.529995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:22.529995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:22.529995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:22.529995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:22.529995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:22.529995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:22.529995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:22.529995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:22.529995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:22.529995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:42:22.529995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:22.529995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:42:22.529995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:22.529995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 458/911] 10.222.0.78 () {56 vars in 1114 bytes} [Wed May 25 14:42:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:22.534278 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d53cd5-8bb9-401a-b3af-ccc4f84cbe53 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:22.563735 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:22.609422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb0f1f-d762-4516-8380-4892500c6ba8 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:22.663226 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a3689-4db4-49d7-9c93-fe7373370230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:22.664845 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a3689-4db4-49d7-9c93-fe7373370230 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:22.664845 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a3689-4db4-49d7-9c93-fe7373370230 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:22.665287 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a3689-4db4-49d7-9c93-fe7373370230 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:22.666914 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a3689-4db4-49d7-9c93-fe7373370230 None admin] RBAC: Adding query filter params (interface=public, service_id=8b0d060592e34978aa5e4453afa2237d, region_id=RegionOne) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:22.667491 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a3689-4db4-49d7-9c93-fe7373370230 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:22.671078 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a3689-4db4-49d7-9c93-fe7373370230 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:22.687750 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 454/912] 10.222.0.78 () {56 vars in 1251 bytes} [Wed May 25 14:42:22 2022] GET /identity/v3/endpoints?service_id=8b0d060592e34978aa5e4453afa2237d&interface=public®ion_id=RegionOne => generated 194 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:22.687750 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a3689-4db4-49d7-9c93-fe7373370230 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:22.704823 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d53cd5-8bb9-401a-b3af-ccc4f84cbe53 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:22.747277 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67fd81-5342-4c3d-a50e-9b4aab93406f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:22.748470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67fd81-5342-4c3d-a50e-9b4aab93406f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:22.748964 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67fd81-5342-4c3d-a50e-9b4aab93406f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:22.749468 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67fd81-5342-4c3d-a50e-9b4aab93406f None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:22.751300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67fd81-5342-4c3d-a50e-9b4aab93406f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:22.751865 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67fd81-5342-4c3d-a50e-9b4aab93406f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:22.754698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67fd81-5342-4c3d-a50e-9b4aab93406f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:22.769581 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:22.786129 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 459/913] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:42:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:22.786129 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67fd81-5342-4c3d-a50e-9b4aab93406f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:22.855840 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a3689-4db4-49d7-9c93-fe7373370230 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:22.916331 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:22.926888 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191dc39-b621-4477-98cb-f485e256cc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:22.927959 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191dc39-b621-4477-98cb-f485e256cc56 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:22.928465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191dc39-b621-4477-98cb-f485e256cc56 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:22.928901 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191dc39-b621-4477-98cb-f485e256cc56 None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:22.930064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5191dc39-b621-4477-98cb-f485e256cc56 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:22.930519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5191dc39-b621-4477-98cb-f485e256cc56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:22.932871 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5191dc39-b621-4477-98cb-f485e256cc56 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:22.943695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 455/914] 10.222.0.78 () {56 vars in 1114 bytes} [Wed May 25 14:42:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:22.945198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191dc39-b621-4477-98cb-f485e256cc56 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:22.956562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67fd81-5342-4c3d-a50e-9b4aab93406f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:22.999876 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2e466-b31a-40b1-88db-afeb0317e152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:23.000566 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2e466-b31a-40b1-88db-afeb0317e152 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:23.000851 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2e466-b31a-40b1-88db-afeb0317e152 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:23.001131 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2e466-b31a-40b1-88db-afeb0317e152 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:23.001830 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2e466-b31a-40b1-88db-afeb0317e152 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:23.003312 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2e466-b31a-40b1-88db-afeb0317e152 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:23.014865 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 460/915] 10.222.0.78 () {56 vars in 1087 bytes} [Wed May 25 14:42:22 2022] GET /identity/v3/services => generated 1497 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:42:23.016572 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2e466-b31a-40b1-88db-afeb0317e152 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:23.152584 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191dc39-b621-4477-98cb-f485e256cc56 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:23.172338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:23.221537 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dae390-563c-4abc-bc0a-83d16bf79d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:23.222661 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dae390-563c-4abc-bc0a-83d16bf79d03 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:23.223097 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dae390-563c-4abc-bc0a-83d16bf79d03 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:23.223526 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dae390-563c-4abc-bc0a-83d16bf79d03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:23.225143 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dae390-563c-4abc-bc0a-83d16bf79d03 None admin] RBAC: Adding query filter params (role.id=074ec0dc23ca41fbb4833f8857859512, scope.project.id=908a13d2b49544fbaf760b07e213bb7d, user.id=b2935579c8554bed8b1e211c47548ac5) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:23.225705 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dae390-563c-4abc-bc0a-83d16bf79d03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:23.228206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dae390-563c-4abc-bc0a-83d16bf79d03 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:23.248168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 456/916] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:42:22 2022] GET /identity/v3/role_assignments?user.id=b2935579c8554bed8b1e211c47548ac5&scope.project.id=908a13d2b49544fbaf760b07e213bb7d&role.id=074ec0dc23ca41fbb4833f8857859512 => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:23.265521 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2e466-b31a-40b1-88db-afeb0317e152 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:23.314642 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a225d-cf2c-443d-b64a-1e09d0748730 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:23.315169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a225d-cf2c-443d-b64a-1e09d0748730 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:23.315553 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a225d-cf2c-443d-b64a-1e09d0748730 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:23.318111 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 457/917] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:23.327006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9cc87-5b1d-4d39-8bfd-05d8db2b04d9 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:23.327546 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9cc87-5b1d-4d39-8bfd-05d8db2b04d9 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:23.327977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9cc87-5b1d-4d39-8bfd-05d8db2b04d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:23.339046 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d457e-a6b1-4078-99ab-5a0317966886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:23.339916 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d457e-a6b1-4078-99ab-5a0317966886 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:23.340882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d457e-a6b1-4078-99ab-5a0317966886 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:23.341272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d457e-a6b1-4078-99ab-5a0317966886 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:23.342207 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d457e-a6b1-4078-99ab-5a0317966886 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:23.342637 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d457e-a6b1-4078-99ab-5a0317966886 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:23.344293 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d457e-a6b1-4078-99ab-5a0317966886 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:23.350455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-31b9cc87-5b1d-4d39-8bfd-05d8db2b04d9 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:23.352130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-31b9cc87-5b1d-4d39-8bfd-05d8db2b04d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:23.356007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 461/918] 10.222.0.78 () {56 vars in 1114 bytes} [Wed May 25 14:42:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:23.357902 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d457e-a6b1-4078-99ab-5a0317966886 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:23.362235 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:23.476325 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 458/919] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:42:23.502437 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b9cc87-5b1d-4d39-8bfd-05d8db2b04d9 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:23.520098 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d457e-a6b1-4078-99ab-5a0317966886 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:23.572139 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c58a8-f9a1-462f-a634-4080ce660ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:23.572940 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c58a8-f9a1-462f-a634-4080ce660ae3 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:23.573238 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c58a8-f9a1-462f-a634-4080ce660ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:23.573617 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c58a8-f9a1-462f-a634-4080ce660ae3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:23.574476 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c58a8-f9a1-462f-a634-4080ce660ae3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:23.576970 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c58a8-f9a1-462f-a634-4080ce660ae3 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:23.610060 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 462/920] 10.222.0.78 () {60 vars in 1144 bytes} [Wed May 25 14:42:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:42:23.684869 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b9cc87-5b1d-4d39-8bfd-05d8db2b04d9 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:23.702497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:23.760460 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f7916-80c6-419c-8da7-f93c2c52f248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:23.761888 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f7916-80c6-419c-8da7-f93c2c52f248 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:23.762441 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f7916-80c6-419c-8da7-f93c2c52f248 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:23.762929 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f7916-80c6-419c-8da7-f93c2c52f248 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:23.764283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62f7916-80c6-419c-8da7-f93c2c52f248 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:23.767046 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62f7916-80c6-419c-8da7-f93c2c52f248 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:23.792376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-f62f7916-80c6-419c-8da7-f93c2c52f248 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:42:23.792376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:23.792376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:42:23.792376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:42:23.792376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:42:23.792376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:42:23.792376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:42:23.792376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:23.792376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:42:23.792376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:23.792376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:42:23.792376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:23.792376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:23.792376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:23.792376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:23.792376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:23.792376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:23.794076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:23.794076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:23.794076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:23.794076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:23.794076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:23.794076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:23.794076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:42:23.794076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:42:23.794076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:42:23.794076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:42:23.794076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:23.794076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:23.794076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:23.794076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:23.794076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:23.794076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:23.794915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:23.794915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:23.794915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:23.794915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:23.794915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:23.794915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:23.794915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:23.794915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:23.794915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:23.794915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:23.794915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:42:23.794915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:23.794915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:42:23.794915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:23.796296 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 459/921] 10.222.0.78 () {56 vars in 1126 bytes} [Wed May 25 14:42:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:23.806562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c58a8-f9a1-462f-a634-4080ce660ae3 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:23.921276 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:23.973569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c58a8-f9a1-462f-a634-4080ce660ae3 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:24.023474 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97d58b-638c-4af1-839a-a6feb5cc8a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:24.024083 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97d58b-638c-4af1-839a-a6feb5cc8a03 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:24.024371 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97d58b-638c-4af1-839a-a6feb5cc8a03 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:24.024653 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97d58b-638c-4af1-839a-a6feb5cc8a03 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:24.025377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97d58b-638c-4af1-839a-a6feb5cc8a03 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:24.025690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97d58b-638c-4af1-839a-a6feb5cc8a03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:24.027150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97d58b-638c-4af1-839a-a6feb5cc8a03 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:24.034027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 463/922] 10.222.0.78 () {56 vars in 1122 bytes} [Wed May 25 14:42:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:24.040323 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f7916-80c6-419c-8da7-f93c2c52f248 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:24.165063 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f7916-80c6-419c-8da7-f93c2c52f248 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:24.209550 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec80bde-1eb5-45b8-9de8-05b6644440b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:24.210245 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec80bde-1eb5-45b8-9de8-05b6644440b8 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:24.210526 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec80bde-1eb5-45b8-9de8-05b6644440b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:24.210798 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec80bde-1eb5-45b8-9de8-05b6644440b8 None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:24.211511 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec80bde-1eb5-45b8-9de8-05b6644440b8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:24.211814 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec80bde-1eb5-45b8-9de8-05b6644440b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:24.213210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec80bde-1eb5-45b8-9de8-05b6644440b8 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:24.219806 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 460/923] 10.222.0.78 () {56 vars in 1122 bytes} [Wed May 25 14:42:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:24.225691 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97d58b-638c-4af1-839a-a6feb5cc8a03 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:24.339496 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97d58b-638c-4af1-839a-a6feb5cc8a03 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:24.369877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:24.381624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d7598-e1e8-4be7-8513-40f5e057a895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:24.383081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d7598-e1e8-4be7-8513-40f5e057a895 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:24.384152 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d7598-e1e8-4be7-8513-40f5e057a895 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:24.385386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d7598-e1e8-4be7-8513-40f5e057a895 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:24.387301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d7598-e1e8-4be7-8513-40f5e057a895 None admin] RBAC: Adding query filter params (interface=public, service_id=7e537b1fb16646e695c4e97bac8b6b3a, region_id=RegionOne) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:24.388076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d7598-e1e8-4be7-8513-40f5e057a895 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:24.390492 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d7598-e1e8-4be7-8513-40f5e057a895 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:24.398149 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 464/924] 10.222.0.78 () {56 vars in 1251 bytes} [Wed May 25 14:42:24 2022] GET /identity/v3/endpoints?service_id=7e537b1fb16646e695c4e97bac8b6b3a&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:24.404372 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec80bde-1eb5-45b8-9de8-05b6644440b8 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:24.440328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:24.523338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec80bde-1eb5-45b8-9de8-05b6644440b8 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:24.587450 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6c3e1-12a6-4f05-9ee2-40cde3c0bca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:24.589085 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6c3e1-12a6-4f05-9ee2-40cde3c0bca7 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:24.589919 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6c3e1-12a6-4f05-9ee2-40cde3c0bca7 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:24.591362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6c3e1-12a6-4f05-9ee2-40cde3c0bca7 None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:24.592958 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d6c3e1-12a6-4f05-9ee2-40cde3c0bca7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:24.595784 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d6c3e1-12a6-4f05-9ee2-40cde3c0bca7 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:24.609052 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 461/925] 10.222.0.78 () {56 vars in 1087 bytes} [Wed May 25 14:42:24 2022] GET /identity/v3/services => generated 1497 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:42:25.561469 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc707fc-b249-4a36-98dc-b827cdd21aa9 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:25.562052 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc707fc-b249-4a36-98dc-b827cdd21aa9 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:25.562326 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc707fc-b249-4a36-98dc-b827cdd21aa9 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:25.564092 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 465/926] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:25.569675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8720fea-d18f-4b26-a260-a3b088575776 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:25.570163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8720fea-d18f-4b26-a260-a3b088575776 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:25.570462 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8720fea-d18f-4b26-a260-a3b088575776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:25.589129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-f8720fea-d18f-4b26-a260-a3b088575776 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:25.590962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f8720fea-d18f-4b26-a260-a3b088575776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:25.664665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 462/927] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:25.688619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc707fc-b249-4a36-98dc-b827cdd21aa9 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:25.710373 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd240f0-b42c-47bb-a319-6e1bebcbe3a6 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:25.710707 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd240f0-b42c-47bb-a319-6e1bebcbe3a6 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:25.710930 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd240f0-b42c-47bb-a319-6e1bebcbe3a6 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:25.712620 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 463/928] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:42:25.718618 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e430d1c-c90f-49df-98dd-50f0d87f9c06 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:25.718901 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e430d1c-c90f-49df-98dd-50f0d87f9c06 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:25.719177 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e430d1c-c90f-49df-98dd-50f0d87f9c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:25.737527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-7e430d1c-c90f-49df-98dd-50f0d87f9c06 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:25.738781 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-7e430d1c-c90f-49df-98dd-50f0d87f9c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:25.764544 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:25.821250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc707fc-b249-4a36-98dc-b827cdd21aa9 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:25.842122 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 464/929] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:25.864929 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e430d1c-c90f-49df-98dd-50f0d87f9c06 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:25.868353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf5dfd-a170-4ef9-9736-44e9b21e21fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:25.869260 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf5dfd-a170-4ef9-9736-44e9b21e21fb None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:25.869610 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf5dfd-a170-4ef9-9736-44e9b21e21fb None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:25.869952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf5dfd-a170-4ef9-9736-44e9b21e21fb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:25.870735 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf5dfd-a170-4ef9-9736-44e9b21e21fb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:25.872876 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf5dfd-a170-4ef9-9736-44e9b21e21fb None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:25.914624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-cfcf5dfd-a170-4ef9-9736-44e9b21e21fb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:42:25.914624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:25.914624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:42:25.914624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:42:25.914624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:42:25.914624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:42:25.914624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:42:25.914624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:25.914624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:42:25.914624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:25.914624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:42:25.914624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:25.914624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:25.914624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:25.914624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:25.914624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:25.915625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:25.915625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:25.915625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:25.915625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:25.915625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:25.915625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:25.915625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:25.915625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:42:25.915625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:42:25.915625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:42:25.915625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:42:25.915625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:25.915625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:25.915625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:25.915625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:25.915625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:25.916478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:25.916478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:25.916478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:25.916478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:25.916478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:25.916478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:25.916478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:25.916478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:25.916478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:25.916478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:25.916478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:25.916478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:42:25.916478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:25.916478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:42:25.916478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:25.917795 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 466/930] 10.222.0.78 () {56 vars in 1132 bytes} [Wed May 25 14:42:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:25.927196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf5dfd-a170-4ef9-9736-44e9b21e21fb None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:26.011369 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e430d1c-c90f-49df-98dd-50f0d87f9c06 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:26.022173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:26.058526 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ed3f7-97be-42b2-bc15-f0d691478541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:26.059749 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ed3f7-97be-42b2-bc15-f0d691478541 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:26.060054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ed3f7-97be-42b2-bc15-f0d691478541 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:26.060329 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ed3f7-97be-42b2-bc15-f0d691478541 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:26.075282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ed3f7-97be-42b2-bc15-f0d691478541 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:42:26.084378 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf5dfd-a170-4ef9-9736-44e9b21e21fb None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:26.085685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-c48ed3f7-97be-42b2-bc15-f0d691478541 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:42:26.085685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:26.085685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:26.085685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:26.085685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:26.085685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:26.085685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:26.085685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:26.085685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:26.085685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:26.085685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:26.085685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:26.085685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:42:26.085685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:42:26.085685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:42:26.085685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:42:26.086639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:42:26.086639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:42:26.086639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:42:26.086639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:42:26.086639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:26.086639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:26.086639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:26.086639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:26.086639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:26.086639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:26.086639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:26.086639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:26.086639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:26.086639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:26.086639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:26.086639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:26.087493 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:26.087493 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:26.087493 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:26.087493 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:26.087493 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:42:26.087493 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:42:26.087493 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:42:26.087493 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:42:26.087493 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:42:26.087493 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:42:26.087493 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:42:26.087493 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:26.090178 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 465/931] 10.222.0.78 () {56 vars in 1120 bytes} [Wed May 25 14:42:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:26.095200 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ed3f7-97be-42b2-bc15-f0d691478541 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:26.121873 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:26.140439 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d510f-37a1-4ac1-bd52-5ba5e8426255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:26.141430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d510f-37a1-4ac1-bd52-5ba5e8426255 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:26.141864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d510f-37a1-4ac1-bd52-5ba5e8426255 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:26.142191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d510f-37a1-4ac1-bd52-5ba5e8426255 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:26.142904 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d510f-37a1-4ac1-bd52-5ba5e8426255 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:26.143216 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d510f-37a1-4ac1-bd52-5ba5e8426255 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:26.144604 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d510f-37a1-4ac1-bd52-5ba5e8426255 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:26.149792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 467/932] 10.222.0.78 () {56 vars in 1126 bytes} [Wed May 25 14:42:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:26.155463 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702e50b-3250-411f-95d4-123010df091a None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:26.155755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702e50b-3250-411f-95d4-123010df091a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:26.156039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702e50b-3250-411f-95d4-123010df091a None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:26.157481 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 468/933] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:26.159140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702e50b-3250-411f-95d4-123010df091a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:26.219565 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ed3f7-97be-42b2-bc15-f0d691478541 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:26.256576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed6eef-f3a3-4e9e-9715-a755a024f6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:26.257241 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed6eef-f3a3-4e9e-9715-a755a024f6dd None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:26.257674 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed6eef-f3a3-4e9e-9715-a755a024f6dd None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:26.257782 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed6eef-f3a3-4e9e-9715-a755a024f6dd None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:26.258814 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed6eef-f3a3-4e9e-9715-a755a024f6dd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:26.258971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed6eef-f3a3-4e9e-9715-a755a024f6dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:26.260333 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed6eef-f3a3-4e9e-9715-a755a024f6dd None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:26.274109 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 466/934] 10.222.0.78 () {56 vars in 1115 bytes} [Wed May 25 14:42:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:26.274109 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3611ae-16b6-4f9c-89a0-30c3f2575c70 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:26.274109 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3611ae-16b6-4f9c-89a0-30c3f2575c70 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:26.274109 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3611ae-16b6-4f9c-89a0-30c3f2575c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:26.275492 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702e50b-3250-411f-95d4-123010df091a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:26.290040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-4a3611ae-16b6-4f9c-89a0-30c3f2575c70 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:26.291290 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-4a3611ae-16b6-4f9c-89a0-30c3f2575c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:26.295963 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:26.317797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6c70e-d335-45bc-a906-e0f2f72a966b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:26.318585 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6c70e-d335-45bc-a906-e0f2f72a966b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:26.318880 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6c70e-d335-45bc-a906-e0f2f72a966b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:26.319164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6c70e-d335-45bc-a906-e0f2f72a966b None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:26.319873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6c70e-d335-45bc-a906-e0f2f72a966b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:26.320180 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6c70e-d335-45bc-a906-e0f2f72a966b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:26.321578 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6c70e-d335-45bc-a906-e0f2f72a966b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:26.326739 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 469/935] 10.222.0.78 () {56 vars in 1126 bytes} [Wed May 25 14:42:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:26.327472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6c70e-d335-45bc-a906-e0f2f72a966b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:26.370366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 467/936] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:26.372121 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3611ae-16b6-4f9c-89a0-30c3f2575c70 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:26.385761 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:26.451231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6c70e-d335-45bc-a906-e0f2f72a966b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:26.499805 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3611ae-16b6-4f9c-89a0-30c3f2575c70 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:26.514917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23ab86-94b1-4056-93ed-fc8231893106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:26.515737 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23ab86-94b1-4056-93ed-fc8231893106 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:26.516025 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23ab86-94b1-4056-93ed-fc8231893106 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:26.516306 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23ab86-94b1-4056-93ed-fc8231893106 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:26.534968 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23ab86-94b1-4056-93ed-fc8231893106 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:26.536869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23ab86-94b1-4056-93ed-fc8231893106 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:26.554131 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f413466-d077-4149-90ec-4833a454001c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:26.555042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f413466-d077-4149-90ec-4833a454001c None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:26.555324 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f413466-d077-4149-90ec-4833a454001c None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:26.555749 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f413466-d077-4149-90ec-4833a454001c None admin] PATH_INFO: `/v3/endpoints` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:26.556529 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f413466-d077-4149-90ec-4833a454001c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:26.558344 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f413466-d077-4149-90ec-4833a454001c None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:26.575300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-6f23ab86-94b1-4056-93ed-fc8231893106 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:26.575300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:26.575300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:26.575300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:26.575300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:26.575300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:26.575300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:26.575300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:26.575300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:26.575300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:26.575300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:26.575300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:26.575300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:26.575300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:26.575300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:26.576672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:26.576672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:26.576672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:26.576672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:26.576672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:26.576672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:26.576672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:26.576672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:26.576672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:26.576672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:26.576672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:26.576672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:26.576672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:26.576672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:26.576672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:26.576672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:26.576672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:26.577655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:26.577655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:26.577655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:26.577655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:26.577889 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 470/937] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:26.579768 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23ab86-94b1-4056-93ed-fc8231893106 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:26.593049 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 468/938] 10.222.0.78 () {60 vars in 1144 bytes} [Wed May 25 14:42:26 2022] POST /identity/v3/endpoints => generated 347 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:42:26.594728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f413466-d077-4149-90ec-4833a454001c None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:26.641513 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:26.735809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23ab86-94b1-4056-93ed-fc8231893106 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:26.753865 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:26.795983 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee44a54-927b-44ae-ab41-02387db8f1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:26.796662 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee44a54-927b-44ae-ab41-02387db8f1f0 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:26.796947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee44a54-927b-44ae-ab41-02387db8f1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:26.797221 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee44a54-927b-44ae-ab41-02387db8f1f0 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:26.797918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee44a54-927b-44ae-ab41-02387db8f1f0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:26.799520 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee44a54-927b-44ae-ab41-02387db8f1f0 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:26.801067 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: WARNING keystone.server.flask.common [None req-0ee44a54-927b-44ae-ab41-02387db8f1f0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 25 14:42:26.813149 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f413466-d077-4149-90ec-4833a454001c None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:26.820164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.sql.core [None req-0ee44a54-927b-44ae-ab41-02387db8f1f0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 25 14:42:26.820164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 25 14:42:26.820164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [parameters: {'id': 'c77aec215cfa498cb1bf4c6faac2edd5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 25 14:42:26.820164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86693) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 25 14:42:26.829990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-0ee44a54-927b-44ae-ab41-02387db8f1f0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c77aec215cfa498cb1bf4c6faac2edd5.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c77aec215cfa498cb1bf4c6faac2edd5. May 25 14:42:26.829990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:26.829990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 14:42:26.829990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 14:42:26.829990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 14:42:26.829990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 14:42:26.829990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 14:42:26.829990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application result = self._query(query) May 25 14:42:26.829990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 14:42:26.829990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application conn.query(q) May 25 14:42:26.829990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 14:42:26.829990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 14:42:26.829990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 14:42:26.830886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application result.read() May 25 14:42:26.830886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 14:42:26.830886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 14:42:26.830886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 14:42:26.830886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 14:42:26.830886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 14:42:26.830886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 14:42:26.830886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 14:42:26.830886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 14:42:26.830886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 25 14:42:26.830886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:26.830886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 25 14:42:26.830886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:26.830886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:26.830886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 25 14:42:26.830886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 25 14:42:26.830886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 25 14:42:26.830886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return project_ref.to_dict() May 25 14:42:26.831832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 14:42:26.831832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application next(self.gen) May 25 14:42:26.831832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 25 14:42:26.831832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application yield resource May 25 14:42:26.831832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 14:42:26.831832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application next(self.gen) May 25 14:42:26.831832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 25 14:42:26.831832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self.session.rollback() May 25 14:42:26.831832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 25 14:42:26.831832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self.force_reraise() May 25 14:42:26.831832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 25 14:42:26.831832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise self.value May 25 14:42:26.831832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 25 14:42:26.831832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 25 14:42:26.831832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 25 14:42:26.831832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application session.commit() May 25 14:42:26.831832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 25 14:42:26.833003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 25 14:42:26.833003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 25 14:42:26.833003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._prepare_impl() May 25 14:42:26.833003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 25 14:42:26.833003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self.session.flush() May 25 14:42:26.833003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 25 14:42:26.833003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._flush(objects) May 25 14:42:26.833003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 25 14:42:26.833003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 25 14:42:26.833003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 25 14:42:26.833003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application compat.raise_( May 25 14:42:26.833003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 14:42:26.833003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception May 25 14:42:26.833003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 25 14:42:26.833003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application flush_context.execute() May 25 14:42:26.833003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 25 14:42:26.833003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rec.execute(self) May 25 14:42:26.835191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 25 14:42:26.835191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 25 14:42:26.835191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 25 14:42:26.835191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application _emit_insert_statements( May 25 14:42:26.835191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 25 14:42:26.835191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application c = connection._execute_20( May 25 14:42:26.835191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 25 14:42:26.835191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 25 14:42:26.835191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 25 14:42:26.835191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 25 14:42:26.835191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 25 14:42:26.835191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ret = self._execute_context( May 25 14:42:26.835191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 25 14:42:26.835191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 25 14:42:26.835191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 25 14:42:26.838345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 25 14:42:26.838345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 14:42:26.838345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception May 25 14:42:26.838345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 14:42:26.838345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 14:42:26.838345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 14:42:26.838345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 14:42:26.838345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 14:42:26.838345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application result = self._query(query) May 25 14:42:26.838345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 14:42:26.838345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application conn.query(q) May 25 14:42:26.838345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 14:42:26.838345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 14:42:26.838345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 14:42:26.838345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application result.read() May 25 14:42:26.838345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 14:42:26.838345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 14:42:26.841069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 14:42:26.841069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 14:42:26.841069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 14:42:26.841069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 14:42:26.841069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 14:42:26.841069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 14:42:26.841069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 25 14:42:26.841069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 25 14:42:26.841069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [parameters: {'id': 'c77aec215cfa498cb1bf4c6faac2edd5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 25 14:42:26.841069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 25 14:42:26.841069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:26.841069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:42:26.841069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:26.841069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:26.841069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 25 14:42:26.844367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 25 14:42:26.844367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 25 14:42:26.844367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 25 14:42:26.844367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 25 14:42:26.844367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:26.844367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:42:26.844367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:26.844367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:26.844367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:26.844367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:26.844367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:26.844367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:26.844367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:26.844367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:26.844367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:26.844367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:26.844367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:26.846621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:26.846621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 25 14:42:26.846621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 25 14:42:26.846621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:26.846621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:26.846621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 25 14:42:26.846621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.Conflict( May 25 14:42:26.846621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c77aec215cfa498cb1bf4c6faac2edd5. May 25 14:42:26.846621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:26.846621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 471/939] 10.222.0.78 () {60 vars in 1140 bytes} [Wed May 25 14:42:26 2022] POST /identity/v3/projects => generated 262 bytes in 258 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:26.846621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee44a54-927b-44ae-ab41-02387db8f1f0 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:26.859116 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1d215-edb4-4090-a95e-e75125102bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:26.859997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1d215-edb4-4090-a95e-e75125102bd3 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:26.860284 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1d215-edb4-4090-a95e-e75125102bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:26.860563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1d215-edb4-4090-a95e-e75125102bd3 None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:26.861304 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd1d215-edb4-4090-a95e-e75125102bd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:26.861609 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd1d215-edb4-4090-a95e-e75125102bd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:26.863201 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd1d215-edb4-4090-a95e-e75125102bd3 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:26.881194 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 469/940] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:26.888462 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1d215-edb4-4090-a95e-e75125102bd3 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:26.979069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:27.001863 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee44a54-927b-44ae-ab41-02387db8f1f0 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:27.038116 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1d215-edb4-4090-a95e-e75125102bd3 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:27.039947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a18b9-b222-475b-97ab-a8d75dab5fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:27.040838 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a18b9-b222-475b-97ab-a8d75dab5fe5 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:27.041240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a18b9-b222-475b-97ab-a8d75dab5fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:27.044425 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a18b9-b222-475b-97ab-a8d75dab5fe5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:27.051957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a18b9-b222-475b-97ab-a8d75dab5fe5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:27.053932 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a18b9-b222-475b-97ab-a8d75dab5fe5 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:27.070551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-c03a18b9-b222-475b-97ab-a8d75dab5fe5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:27.070551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:27.070551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:27.070551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:27.070551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:27.070551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:27.070551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:27.070551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:27.070551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:27.070551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:27.070551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:27.070551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:27.070551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:42:27.070551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:42:27.070551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:42:27.072432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:42:27.072432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:27.072432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:27.072432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:27.072432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:27.072432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:27.072432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:27.072432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:27.072432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:27.072432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:27.072432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:27.072432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:27.072432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:27.072432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:27.072432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:27.072432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:27.072432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:27.075827 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:42:27.075827 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:42:27.075827 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:42:27.075827 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:42:27.075827 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:42:27.075827 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:42:27.075827 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:27.075827 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:27.075827 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 472/941] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:42:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:27.084134 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a18b9-b222-475b-97ab-a8d75dab5fe5 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:27.091012 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:27.103886 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c2974-304a-4bd2-b884-92e67bc09c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:27.104701 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c2974-304a-4bd2-b884-92e67bc09c76 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:27.104983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c2974-304a-4bd2-b884-92e67bc09c76 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:27.105253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c2974-304a-4bd2-b884-92e67bc09c76 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:27.113454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c2974-304a-4bd2-b884-92e67bc09c76 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:27.114973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c2974-304a-4bd2-b884-92e67bc09c76 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:27.131576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-368c2974-304a-4bd2-b884-92e67bc09c76 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:42:27.131576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:27.131576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:27.131576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:27.131576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:27.131576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:27.131576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:27.131576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:27.131576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:27.131576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:27.131576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:27.131576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:27.131576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:42:27.131576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:42:27.131576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:42:27.131576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:42:27.132716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:27.132716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:27.132716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:42:27.132716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:27.132716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:42:27.132716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:27.132716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:27.132716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:27.132716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:27.132716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:27.132716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:27.132716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:27.132716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:27.132716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:27.132716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:27.132716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:27.132716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:27.132716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:27.133684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:27.133684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:27.133684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:42:27.133684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:42:27.133684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:42:27.133684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:42:27.133684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:42:27.133684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:42:27.133684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:42:27.133684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:27.138079 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 470/942] 10.222.0.78 () {56 vars in 1117 bytes} [Wed May 25 14:42:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:27.139068 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c2974-304a-4bd2-b884-92e67bc09c76 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:27.226349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a18b9-b222-475b-97ab-a8d75dab5fe5 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:27.253841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c2974-304a-4bd2-b884-92e67bc09c76 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:27.267025 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4044a70d-6003-44de-aca6-c0a92d76b088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:27.267703 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044a70d-6003-44de-aca6-c0a92d76b088 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:27.268065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044a70d-6003-44de-aca6-c0a92d76b088 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:27.268374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044a70d-6003-44de-aca6-c0a92d76b088 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:27.269030 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4044a70d-6003-44de-aca6-c0a92d76b088 None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:27.269573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4044a70d-6003-44de-aca6-c0a92d76b088 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:27.270878 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4044a70d-6003-44de-aca6-c0a92d76b088 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:27.291526 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395b24a-ee99-46f9-b8c5-9e934e97ab96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:27.292198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395b24a-ee99-46f9-b8c5-9e934e97ab96 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:27.292502 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395b24a-ee99-46f9-b8c5-9e934e97ab96 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:27.298178 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395b24a-ee99-46f9-b8c5-9e934e97ab96 None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:27.298397 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 473/943] 10.222.0.78 () {56 vars in 1112 bytes} [Wed May 25 14:42:27 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:27.298554 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395b24a-ee99-46f9-b8c5-9e934e97ab96 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:27.298554 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395b24a-ee99-46f9-b8c5-9e934e97ab96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:27.298752 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd23bd-2276-4544-9066-bf6c40bc4224 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:27.298752 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd23bd-2276-4544-9066-bf6c40bc4224 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:27.298752 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd23bd-2276-4544-9066-bf6c40bc4224 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:27.298995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395b24a-ee99-46f9-b8c5-9e934e97ab96 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:27.301534 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 474/944] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:27.305443 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1379a6-af0e-499f-950a-76fb02d47b46 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:27.305443 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1379a6-af0e-499f-950a-76fb02d47b46 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:27.305443 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1379a6-af0e-499f-950a-76fb02d47b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:27.316504 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:27.322876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 471/945] 10.222.0.78 () {56 vars in 1149 bytes} [Wed May 25 14:42:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:27.325614 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-5f1379a6-af0e-499f-950a-76fb02d47b46 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:27.327727 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5f1379a6-af0e-499f-950a-76fb02d47b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:27.329303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395b24a-ee99-46f9-b8c5-9e934e97ab96 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:27.398735 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:27.422459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 475/946] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:27.439586 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1379a6-af0e-499f-950a-76fb02d47b46 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:27.489429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395b24a-ee99-46f9-b8c5-9e934e97ab96 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:27.540975 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f6306-bc35-417b-8a23-e97b90bbcac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:27.542141 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f6306-bc35-417b-8a23-e97b90bbcac2 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:27.542428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f6306-bc35-417b-8a23-e97b90bbcac2 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:27.542703 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f6306-bc35-417b-8a23-e97b90bbcac2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:27.556900 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f6306-bc35-417b-8a23-e97b90bbcac2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:27.558738 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f6306-bc35-417b-8a23-e97b90bbcac2 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:27.571367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1379a6-af0e-499f-950a-76fb02d47b46 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:27.583816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-223f6306-bc35-417b-8a23-e97b90bbcac2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:27.583816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:27.583816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:27.583816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:27.583816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:27.583816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:27.583816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:27.583816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:27.583816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:27.583816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:27.583816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:27.583816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:27.583816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:27.583816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:27.583816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:27.584771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:27.584771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:27.584771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:27.584771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:27.584771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:27.584771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:27.584771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:27.584771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:27.584771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:27.584771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:27.584771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:27.584771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:27.584771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:27.584771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:27.584771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:27.584771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:27.584771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:27.585676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:27.585676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:27.585676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:27.585676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:27.586224 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 472/947] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:27.592126 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f6306-bc35-417b-8a23-e97b90bbcac2 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:27.620005 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da73df3-1cc5-4112-af93-b5e343a456d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:27.620820 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da73df3-1cc5-4112-af93-b5e343a456d8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:27.621099 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da73df3-1cc5-4112-af93-b5e343a456d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:27.621451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da73df3-1cc5-4112-af93-b5e343a456d8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:27.622940 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da73df3-1cc5-4112-af93-b5e343a456d8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:27.623534 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da73df3-1cc5-4112-af93-b5e343a456d8 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:27.626808 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:27.640918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-7da73df3-1cc5-4112-af93-b5e343a456d8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:42:27.640918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:27.640918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:42:27.640918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:42:27.640918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:42:27.640918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:42:27.640918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:42:27.640918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:27.640918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:42:27.640918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:27.640918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:42:27.640918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:27.640918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:27.640918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:27.640918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:27.640918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:27.640918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:27.642069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:27.642069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:27.642069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:27.642069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:27.642069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:27.642069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:27.642069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:42:27.642069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:42:27.642069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:42:27.642069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:42:27.642069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:27.642069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:27.642069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:27.642069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:27.642069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:27.642069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:27.642926 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:27.642926 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:27.642926 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:27.642926 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:27.642926 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:27.642926 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:27.642926 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:27.642926 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:27.642926 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:27.642926 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:27.642926 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:42:27.642926 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:27.642926 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:42:27.642926 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:27.642926 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 476/948] 10.222.0.78 () {56 vars in 1126 bytes} [Wed May 25 14:42:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:27.648696 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da73df3-1cc5-4112-af93-b5e343a456d8 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:27.732605 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:27.743230 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f6306-bc35-417b-8a23-e97b90bbcac2 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:27.788453 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da73df3-1cc5-4112-af93-b5e343a456d8 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:27.795047 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfeced2-19ce-4964-a207-8e0fe2e9d420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:27.796280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfeced2-19ce-4964-a207-8e0fe2e9d420 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:27.796881 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfeced2-19ce-4964-a207-8e0fe2e9d420 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:27.797462 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfeced2-19ce-4964-a207-8e0fe2e9d420 None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:27.798982 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfeced2-19ce-4964-a207-8e0fe2e9d420 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:27.799632 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfeced2-19ce-4964-a207-8e0fe2e9d420 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:27.802669 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfeced2-19ce-4964-a207-8e0fe2e9d420 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:27.821559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 473/949] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:27.831357 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfeced2-19ce-4964-a207-8e0fe2e9d420 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:27.841161 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80545b1-64f6-4da2-9514-aba472771cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:27.841957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80545b1-64f6-4da2-9514-aba472771cad None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:27.842407 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80545b1-64f6-4da2-9514-aba472771cad None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:27.842869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80545b1-64f6-4da2-9514-aba472771cad None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:27.843670 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80545b1-64f6-4da2-9514-aba472771cad None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:27.844065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80545b1-64f6-4da2-9514-aba472771cad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:27.845908 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80545b1-64f6-4da2-9514-aba472771cad None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:27.859726 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 477/950] 10.222.0.78 () {56 vars in 1122 bytes} [Wed May 25 14:42:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:27.861085 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80545b1-64f6-4da2-9514-aba472771cad None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:27.952962 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:27.998958 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfeced2-19ce-4964-a207-8e0fe2e9d420 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:28.000797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80545b1-64f6-4da2-9514-aba472771cad None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:28.035832 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021b86b-320b-45b6-9e56-67b8e8d80514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:28.036725 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021b86b-320b-45b6-9e56-67b8e8d80514 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:28.037001 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021b86b-320b-45b6-9e56-67b8e8d80514 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:28.037268 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021b86b-320b-45b6-9e56-67b8e8d80514 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:28.041413 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98562e-e2c1-4152-81d7-960ef8919b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:28.042393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98562e-e2c1-4152-81d7-960ef8919b37 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:28.042902 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98562e-e2c1-4152-81d7-960ef8919b37 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:28.043983 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98562e-e2c1-4152-81d7-960ef8919b37 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:28.044901 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98562e-e2c1-4152-81d7-960ef8919b37 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:28.045424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98562e-e2c1-4152-81d7-960ef8919b37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:28.045692 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c021b86b-320b-45b6-9e56-67b8e8d80514 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:28.047227 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c021b86b-320b-45b6-9e56-67b8e8d80514 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:28.047359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98562e-e2c1-4152-81d7-960ef8919b37 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:28.058735 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 478/951] 10.222.0.78 () {56 vars in 1122 bytes} [Wed May 25 14:42:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:28.059933 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98562e-e2c1-4152-81d7-960ef8919b37 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:28.062727 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-c021b86b-320b-45b6-9e56-67b8e8d80514 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:28.062727 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:28.062727 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:28.062727 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:28.062727 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:28.062727 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:28.062727 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:28.062727 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:28.062727 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:28.062727 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:28.062727 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:28.062727 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:28.062727 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:42:28.062727 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:42:28.062727 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:42:28.063696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:42:28.063696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:28.063696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:28.063696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:28.063696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:28.063696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:28.063696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:28.063696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:28.063696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:28.063696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:28.063696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:28.063696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:28.063696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:28.063696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:28.063696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:28.063696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:28.063696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:28.064618 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:42:28.064618 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:42:28.064618 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:42:28.064618 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:42:28.064618 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:42:28.064618 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:42:28.064618 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:28.064618 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:28.069264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 474/952] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:42:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:28.070314 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021b86b-320b-45b6-9e56-67b8e8d80514 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:28.087639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:28.177804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021b86b-320b-45b6-9e56-67b8e8d80514 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:28.179552 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98562e-e2c1-4152-81d7-960ef8919b37 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:28.207993 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:28.214370 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753208a8-b3ac-43e4-b2ab-162feef29279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:28.215544 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753208a8-b3ac-43e4-b2ab-162feef29279 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:28.215826 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753208a8-b3ac-43e4-b2ab-162feef29279 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:28.216100 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753208a8-b3ac-43e4-b2ab-162feef29279 None admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:28.216895 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753208a8-b3ac-43e4-b2ab-162feef29279 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:28.217281 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753208a8-b3ac-43e4-b2ab-162feef29279 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:28.219116 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753208a8-b3ac-43e4-b2ab-162feef29279 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:28.220775 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110b280-7914-4489-8e01-fa709438de57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:28.221766 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110b280-7914-4489-8e01-fa709438de57 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:28.222302 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110b280-7914-4489-8e01-fa709438de57 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:28.222833 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110b280-7914-4489-8e01-fa709438de57 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:28.223818 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3110b280-7914-4489-8e01-fa709438de57 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:28.225464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3110b280-7914-4489-8e01-fa709438de57 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:28.237450 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 475/953] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:42:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:28.238480 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753208a8-b3ac-43e4-b2ab-162feef29279 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:28.251374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 479/954] 10.222.0.78 () {60 vars in 1144 bytes} [Wed May 25 14:42:28 2022] POST /identity/v3/endpoints => generated 349 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:42:28.340728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:28.369214 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753208a8-b3ac-43e4-b2ab-162feef29279 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:28.414971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48211677-d90c-481f-9a1b-bdfa48365621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:28.415874 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48211677-d90c-481f-9a1b-bdfa48365621 None admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:28.416151 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48211677-d90c-481f-9a1b-bdfa48365621 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:28.416422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48211677-d90c-481f-9a1b-bdfa48365621 None admin] PATH_INFO: `/v3/projects/908a13d2b49544fbaf760b07e213bb7d/users/b2935579c8554bed8b1e211c47548ac5/roles/074ec0dc23ca41fbb4833f8857859512` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:28.419060 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48211677-d90c-481f-9a1b-bdfa48365621 None admin] RBAC: Authorizing `identity:create_grant(project_id=908a13d2b49544fbaf760b07e213bb7d, user_id=b2935579c8554bed8b1e211c47548ac5, role_id=074ec0dc23ca41fbb4833f8857859512)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:28.420718 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48211677-d90c-481f-9a1b-bdfa48365621 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:28.439338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 476/955] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:42:28 2022] PUT /identity/v3/projects/908a13d2b49544fbaf760b07e213bb7d/users/b2935579c8554bed8b1e211c47548ac5/roles/074ec0dc23ca41fbb4833f8857859512 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:29.063385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e99052-a8f9-4ff8-b9d0-dc7983887d9c None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:29.063953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e99052-a8f9-4ff8-b9d0-dc7983887d9c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:29.063953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e99052-a8f9-4ff8-b9d0-dc7983887d9c None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:29.065816 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 480/956] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:29.071952 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96462859-40e1-4022-83c0-f2c87c9723b4 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:29.072936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96462859-40e1-4022-83c0-f2c87c9723b4 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:29.073517 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96462859-40e1-4022-83c0-f2c87c9723b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:29.093616 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-96462859-40e1-4022-83c0-f2c87c9723b4 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:29.095300 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-96462859-40e1-4022-83c0-f2c87c9723b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:29.178215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 477/957] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:29.197408 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e99052-a8f9-4ff8-b9d0-dc7983887d9c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:29.327635 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e99052-a8f9-4ff8-b9d0-dc7983887d9c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:29.371431 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797ed68-d4cc-400e-abf1-e1c6bf8b4986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:29.372249 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797ed68-d4cc-400e-abf1-e1c6bf8b4986 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:29.372534 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797ed68-d4cc-400e-abf1-e1c6bf8b4986 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:29.372811 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797ed68-d4cc-400e-abf1-e1c6bf8b4986 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:29.373474 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797ed68-d4cc-400e-abf1-e1c6bf8b4986 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:29.374936 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797ed68-d4cc-400e-abf1-e1c6bf8b4986 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:29.387848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-0797ed68-d4cc-400e-abf1-e1c6bf8b4986 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:42:29.387848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:29.387848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:42:29.387848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:42:29.387848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:42:29.387848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:42:29.387848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:42:29.387848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:29.387848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:42:29.387848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:29.387848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:42:29.387848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:29.387848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:29.387848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:29.387848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:29.387848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:29.387848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:29.389003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:29.389003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:29.389003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:29.389003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:29.389003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:29.389003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:29.389003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:42:29.389003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:42:29.389003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:42:29.389003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:42:29.389003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:29.389003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:29.389003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:29.389003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:29.389003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:29.389003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:29.390073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:29.390073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:29.390073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:29.390073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:29.390073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:29.390073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:29.390073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:29.390073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:29.390073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:29.390073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:29.390073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:42:29.390073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:29.390073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:42:29.390073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:29.390073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 481/958] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:42:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:29.397802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96462859-40e1-4022-83c0-f2c87c9723b4 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:29.463058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:29.507137 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96462859-40e1-4022-83c0-f2c87c9723b4 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:29.546035 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae26c3-645c-4d6d-a052-9c8d71823b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:29.546735 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae26c3-645c-4d6d-a052-9c8d71823b02 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:29.547017 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae26c3-645c-4d6d-a052-9c8d71823b02 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:29.547293 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae26c3-645c-4d6d-a052-9c8d71823b02 None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:29.548035 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae26c3-645c-4d6d-a052-9c8d71823b02 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:29.548346 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae26c3-645c-4d6d-a052-9c8d71823b02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:29.549772 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae26c3-645c-4d6d-a052-9c8d71823b02 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:29.556006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 478/959] 10.222.0.78 () {56 vars in 1112 bytes} [Wed May 25 14:42:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:29.562494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797ed68-d4cc-400e-abf1-e1c6bf8b4986 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:29.625008 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:29.694626 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797ed68-d4cc-400e-abf1-e1c6bf8b4986 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:29.731217 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d0675-e682-4f3d-a006-88e47a36720b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:29.731893 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d0675-e682-4f3d-a006-88e47a36720b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:29.732173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d0675-e682-4f3d-a006-88e47a36720b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:29.732442 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d0675-e682-4f3d-a006-88e47a36720b None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:29.733137 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d0675-e682-4f3d-a006-88e47a36720b None admin] RBAC: Adding query filter params (type=compute) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:29.733437 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d0675-e682-4f3d-a006-88e47a36720b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:29.734939 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d0675-e682-4f3d-a006-88e47a36720b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:29.740128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 482/960] 10.222.0.78 () {56 vars in 1112 bytes} [Wed May 25 14:42:29 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:29.758681 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff293e-e922-45ee-8c49-6ce48a07330d None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:29.759316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff293e-e922-45ee-8c49-6ce48a07330d None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:29.759822 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff293e-e922-45ee-8c49-6ce48a07330d None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:29.761895 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 479/961] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:29.767254 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f99d7-93a4-4916-a670-c6ea23bd2cbe None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:29.767544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f99d7-93a4-4916-a670-c6ea23bd2cbe None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:29.767822 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f99d7-93a4-4916-a670-c6ea23bd2cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:29.787604 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-3d3f99d7-93a4-4916-a670-c6ea23bd2cbe None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:29.789275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3d3f99d7-93a4-4916-a670-c6ea23bd2cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:29.921709 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 483/962] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:29.946465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff293e-e922-45ee-8c49-6ce48a07330d None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:30.117131 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff293e-e922-45ee-8c49-6ce48a07330d None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:30.147457 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:30.158404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca87181-de6e-4dd6-aad5-66f8d109497b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:30.160107 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca87181-de6e-4dd6-aad5-66f8d109497b None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:30.160723 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca87181-de6e-4dd6-aad5-66f8d109497b None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:30.161234 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca87181-de6e-4dd6-aad5-66f8d109497b None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:30.162269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca87181-de6e-4dd6-aad5-66f8d109497b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:30.164048 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca87181-de6e-4dd6-aad5-66f8d109497b None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:30.165667 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: WARNING keystone.server.flask.common [None req-1ca87181-de6e-4dd6-aad5-66f8d109497b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 25 14:42:30.181123 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.sql.core [None req-1ca87181-de6e-4dd6-aad5-66f8d109497b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 25 14:42:30.181123 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 25 14:42:30.181123 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [parameters: {'user_id': 'efb52c9a50414d3088c1c0565bbac596', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 25 14:42:30.181123 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86692) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 25 14:42:30.208456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-1ca87181-de6e-4dd6-aad5-66f8d109497b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 25 14:42:30.208456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:30.208456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 14:42:30.208456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 14:42:30.208456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 14:42:30.208456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 14:42:30.208456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 14:42:30.208456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application result = self._query(query) May 25 14:42:30.208456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 14:42:30.208456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application conn.query(q) May 25 14:42:30.208456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 14:42:30.208456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 14:42:30.208456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 14:42:30.208456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application result.read() May 25 14:42:30.208456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 14:42:30.209393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 14:42:30.209393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 14:42:30.209393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 14:42:30.209393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 14:42:30.209393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 14:42:30.209393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 14:42:30.209393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 14:42:30.209393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 25 14:42:30.209393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:30.209393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 25 14:42:30.209393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:30.209393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:30.209393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 25 14:42:30.209393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 25 14:42:30.209393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 25 14:42:30.209393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 25 14:42:30.209393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 14:42:30.209393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application next(self.gen) May 25 14:42:30.210378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 25 14:42:30.210378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application yield resource May 25 14:42:30.210378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 14:42:30.210378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application next(self.gen) May 25 14:42:30.210378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 25 14:42:30.210378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self.session.rollback() May 25 14:42:30.210378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 25 14:42:30.210378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self.force_reraise() May 25 14:42:30.210378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 25 14:42:30.210378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise self.value May 25 14:42:30.210378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 25 14:42:30.210378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 25 14:42:30.210378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 25 14:42:30.210378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application session.commit() May 25 14:42:30.210378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 25 14:42:30.210378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 25 14:42:30.210378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 25 14:42:30.211262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._prepare_impl() May 25 14:42:30.211262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 25 14:42:30.211262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self.session.flush() May 25 14:42:30.211262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 25 14:42:30.211262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._flush(objects) May 25 14:42:30.211262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 25 14:42:30.211262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 25 14:42:30.211262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 25 14:42:30.211262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application compat.raise_( May 25 14:42:30.211262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 14:42:30.211262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception May 25 14:42:30.211262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 25 14:42:30.211262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application flush_context.execute() May 25 14:42:30.211262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 25 14:42:30.211262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rec.execute(self) May 25 14:42:30.211262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 25 14:42:30.211262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 25 14:42:30.213563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 25 14:42:30.213563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application _emit_insert_statements( May 25 14:42:30.213563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 25 14:42:30.213563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application result = connection._execute_20( May 25 14:42:30.213563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 25 14:42:30.213563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 25 14:42:30.213563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 25 14:42:30.213563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 25 14:42:30.213563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 25 14:42:30.213563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ret = self._execute_context( May 25 14:42:30.213563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 25 14:42:30.213563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 25 14:42:30.213563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 25 14:42:30.213563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 25 14:42:30.213563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 14:42:30.213563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception May 25 14:42:30.222404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 14:42:30.222404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 14:42:30.222404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 14:42:30.222404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 14:42:30.222404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 14:42:30.222404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application result = self._query(query) May 25 14:42:30.222404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 14:42:30.222404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application conn.query(q) May 25 14:42:30.222404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 14:42:30.222404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 14:42:30.222404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 14:42:30.222404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application result.read() May 25 14:42:30.222404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 14:42:30.222404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 14:42:30.222404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 14:42:30.222404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 14:42:30.224393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 14:42:30.224393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 14:42:30.224393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 14:42:30.224393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 14:42:30.224393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 25 14:42:30.224393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 25 14:42:30.224393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [parameters: {'user_id': 'efb52c9a50414d3088c1c0565bbac596', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 25 14:42:30.224393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 25 14:42:30.224393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:30.224393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:42:30.224393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:30.224393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:30.224393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:30.224393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:30.224393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:30.226282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:30.226282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:30.226282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:30.226282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:30.226282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:30.226282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:30.226282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:30.226282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 25 14:42:30.226282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 25 14:42:30.226282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:30.226282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:30.226282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:42:30.226282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:30.226282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:42:30.226282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:30.226282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 25 14:42:30.226282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 25 14:42:30.228331 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 25 14:42:30.228331 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 25 14:42:30.228331 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 25 14:42:30.228331 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 25 14:42:30.228331 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 25 14:42:30.228331 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:30.228331 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 480/963] 10.222.0.78 () {60 vars in 1131 bytes} [Wed May 25 14:42:29 2022] POST /identity/v3/users => generated 160 bytes in 265 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:30.229238 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f99d7-93a4-4916-a670-c6ea23bd2cbe None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:30.274262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:30.354519 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f99d7-93a4-4916-a670-c6ea23bd2cbe None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:30.396107 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a460eec-7201-4dd6-9f41-8ca1f165ac45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:30.396836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a460eec-7201-4dd6-9f41-8ca1f165ac45 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:30.397117 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a460eec-7201-4dd6-9f41-8ca1f165ac45 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:30.397390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a460eec-7201-4dd6-9f41-8ca1f165ac45 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:30.405604 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a460eec-7201-4dd6-9f41-8ca1f165ac45 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:30.407095 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a460eec-7201-4dd6-9f41-8ca1f165ac45 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:30.423160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-5a460eec-7201-4dd6-9f41-8ca1f165ac45 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 14:42:30.423160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:30.423160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:30.423160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:30.423160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:30.423160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:30.423160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:30.423160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:30.423160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:30.423160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:30.423160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:30.423160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:30.423160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:42:30.423160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:42:30.423160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:42:30.423160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:42:30.425451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:30.425451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:30.425451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:42:30.425451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:30.425451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:42:30.425451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:30.425451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:30.425451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:30.425451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:30.425451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:30.425451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:30.425451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:30.425451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:30.425451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:30.425451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:30.425451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:30.425451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:30.425451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:30.426794 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:30.426794 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:30.426794 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:42:30.426794 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:42:30.426794 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:42:30.426794 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:42:30.426794 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:42:30.426794 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:42:30.426794 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 14:42:30.426794 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:30.426794 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 484/964] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:42:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:30.435605 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca87181-de6e-4dd6-aad5-66f8d109497b None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:30.581373 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca87181-de6e-4dd6-aad5-66f8d109497b None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:30.623532 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca4f16-528d-42a8-8592-2ace389a5d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:30.624221 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca4f16-528d-42a8-8592-2ace389a5d97 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:30.624503 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca4f16-528d-42a8-8592-2ace389a5d97 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:30.624786 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca4f16-528d-42a8-8592-2ace389a5d97 None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:30.625607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca4f16-528d-42a8-8592-2ace389a5d97 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:30.625915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca4f16-528d-42a8-8592-2ace389a5d97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:30.627606 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca4f16-528d-42a8-8592-2ace389a5d97 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:30.671751 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 481/965] 10.222.0.78 () {56 vars in 1101 bytes} [Wed May 25 14:42:30 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:30.796986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09264ea0-5312-4df4-abc3-d36cb4351c15 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:30.796986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09264ea0-5312-4df4-abc3-d36cb4351c15 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:30.796986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09264ea0-5312-4df4-abc3-d36cb4351c15 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:30.796986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 485/966] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:30.803686 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc24ffc-bfed-4899-aaa5-7846ff40ccc8 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:30.804016 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc24ffc-bfed-4899-aaa5-7846ff40ccc8 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:30.804244 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc24ffc-bfed-4899-aaa5-7846ff40ccc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:30.823584 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-0fc24ffc-bfed-4899-aaa5-7846ff40ccc8 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:30.826320 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-0fc24ffc-bfed-4899-aaa5-7846ff40ccc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:30.872014 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:30.914040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 482/967] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:30.929663 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09264ea0-5312-4df4-abc3-d36cb4351c15 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:30.994804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7b326-e72c-44c1-9bde-58d36a20720a None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:30.995118 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7b326-e72c-44c1-9bde-58d36a20720a None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:30.995434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7b326-e72c-44c1-9bde-58d36a20720a None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:31.002150 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 483/968] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:31.003275 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1ba60-a668-4bf3-b39f-32ddd0dc2fd5 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:31.003569 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1ba60-a668-4bf3-b39f-32ddd0dc2fd5 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:31.003850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1ba60-a668-4bf3-b39f-32ddd0dc2fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:31.023758 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-d9e1ba60-a668-4bf3-b39f-32ddd0dc2fd5 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:31.025006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d9e1ba60-a668-4bf3-b39f-32ddd0dc2fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:31.069264 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:31.077791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09264ea0-5312-4df4-abc3-d36cb4351c15 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:31.121697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 484/969] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:42:31.136273 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236cffb-d2e6-43b4-b244-abaae92d7a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:31.137187 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236cffb-d2e6-43b4-b244-abaae92d7a41 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:31.137473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236cffb-d2e6-43b4-b244-abaae92d7a41 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:31.137746 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236cffb-d2e6-43b4-b244-abaae92d7a41 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:31.138484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236cffb-d2e6-43b4-b244-abaae92d7a41 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:31.139942 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236cffb-d2e6-43b4-b244-abaae92d7a41 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:31.153840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-9236cffb-d2e6-43b4-b244-abaae92d7a41 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:42:31.153840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:31.153840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:42:31.153840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:42:31.153840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:42:31.153840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:42:31.153840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:42:31.153840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:31.153840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:42:31.153840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:31.153840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:42:31.153840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:31.153840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:31.153840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:31.153840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:31.153840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:31.153840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:31.154952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:31.154952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:31.154952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:31.154952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:31.154952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:31.154952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:31.154952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:42:31.154952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:42:31.154952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:42:31.154952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:42:31.154952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:31.154952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:31.154952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:31.154952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:31.154952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:31.154952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:31.155783 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:31.155783 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:31.155783 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:31.155783 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:31.155783 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:31.155783 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:31.155783 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:31.155783 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:31.155783 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:31.155783 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:31.155783 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:42:31.155783 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:31.155783 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:42:31.155783 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:31.157697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 486/970] 10.222.0.78 () {56 vars in 1126 bytes} [Wed May 25 14:42:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:31.162408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1ba60-a668-4bf3-b39f-32ddd0dc2fd5 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:31.162584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236cffb-d2e6-43b4-b244-abaae92d7a41 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:31.309661 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1ba60-a668-4bf3-b39f-32ddd0dc2fd5 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:31.313670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:31.345076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236cffb-d2e6-43b4-b244-abaae92d7a41 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:31.365560 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6cbc2-5d77-49cc-8b41-5dda451c9e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:31.366434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6cbc2-5d77-49cc-8b41-5dda451c9e40 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:31.366712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6cbc2-5d77-49cc-8b41-5dda451c9e40 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:31.366984 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6cbc2-5d77-49cc-8b41-5dda451c9e40 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:31.386049 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6cbc2-5d77-49cc-8b41-5dda451c9e40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:42:31.394539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-dbb6cbc2-5d77-49cc-8b41-5dda451c9e40 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:42:31.394539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:31.394539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:31.394539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:31.394539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:31.394539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:31.394539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:31.394539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:31.394539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:31.394539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:31.394539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:31.394539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:31.394539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:42:31.394539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:42:31.394539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:42:31.394539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:42:31.398482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:42:31.398482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:42:31.398482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:42:31.398482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:42:31.398482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:31.398482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:31.398482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:31.398482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:31.398482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:31.398482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:31.398482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:31.398482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:31.398482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:31.398482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:31.398482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:31.398482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:31.399338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:31.399338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:31.399338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:31.399338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:31.399338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:42:31.399338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:42:31.399338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:42:31.399338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:42:31.399338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:42:31.399338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:42:31.399338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:42:31.399338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:31.399338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 485/971] 10.222.0.78 () {56 vars in 1120 bytes} [Wed May 25 14:42:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:31.405399 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6cbc2-5d77-49cc-8b41-5dda451c9e40 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:31.430120 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a285e1c-857a-49e0-9819-21ee677738d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:31.430907 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a285e1c-857a-49e0-9819-21ee677738d4 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:31.431257 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a285e1c-857a-49e0-9819-21ee677738d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:31.431769 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a285e1c-857a-49e0-9819-21ee677738d4 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:31.433209 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a285e1c-857a-49e0-9819-21ee677738d4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:31.433209 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a285e1c-857a-49e0-9819-21ee677738d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:31.435334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a285e1c-857a-49e0-9819-21ee677738d4 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:31.446439 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 487/972] 10.222.0.78 () {56 vars in 1122 bytes} [Wed May 25 14:42:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:31.447705 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a285e1c-857a-49e0-9819-21ee677738d4 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:31.483709 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:31.557217 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6cbc2-5d77-49cc-8b41-5dda451c9e40 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:31.581735 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a285e1c-857a-49e0-9819-21ee677738d4 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:31.608558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f69f2-3cb5-49b6-8724-34fac92aeb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:31.609252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f69f2-3cb5-49b6-8724-34fac92aeb6b None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:31.609530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f69f2-3cb5-49b6-8724-34fac92aeb6b None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:31.609802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f69f2-3cb5-49b6-8724-34fac92aeb6b None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:31.610597 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f69f2-3cb5-49b6-8724-34fac92aeb6b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:31.610903 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f69f2-3cb5-49b6-8724-34fac92aeb6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:31.612348 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f69f2-3cb5-49b6-8724-34fac92aeb6b None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:31.633198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 486/973] 10.222.0.78 () {56 vars in 1115 bytes} [Wed May 25 14:42:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:31.634623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f69f2-3cb5-49b6-8724-34fac92aeb6b None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:31.643427 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16113233-3a4d-4d72-b34b-02cf5b7f1f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:31.644474 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16113233-3a4d-4d72-b34b-02cf5b7f1f88 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:31.644984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16113233-3a4d-4d72-b34b-02cf5b7f1f88 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:31.645558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16113233-3a4d-4d72-b34b-02cf5b7f1f88 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:31.646878 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16113233-3a4d-4d72-b34b-02cf5b7f1f88 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:31.647384 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16113233-3a4d-4d72-b34b-02cf5b7f1f88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:31.649899 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16113233-3a4d-4d72-b34b-02cf5b7f1f88 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:31.666085 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 488/974] 10.222.0.78 () {56 vars in 1122 bytes} [Wed May 25 14:42:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:31.669462 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16113233-3a4d-4d72-b34b-02cf5b7f1f88 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:31.690384 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:31.799075 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f69f2-3cb5-49b6-8724-34fac92aeb6b None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:31.806311 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:31.820769 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16113233-3a4d-4d72-b34b-02cf5b7f1f88 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:31.842675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d7235-13cb-4634-b314-4a9bf6f88ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:31.843389 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d7235-13cb-4634-b314-4a9bf6f88ae3 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:31.843741 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d7235-13cb-4634-b314-4a9bf6f88ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:31.844069 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d7235-13cb-4634-b314-4a9bf6f88ae3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:31.858703 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d7235-13cb-4634-b314-4a9bf6f88ae3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:31.860181 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d7235-13cb-4634-b314-4a9bf6f88ae3 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:31.866681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a02d3-6d93-43fc-874b-eef3bf0aff6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:31.867354 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a02d3-6d93-43fc-874b-eef3bf0aff6a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:31.867635 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a02d3-6d93-43fc-874b-eef3bf0aff6a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:31.867904 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a02d3-6d93-43fc-874b-eef3bf0aff6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:31.868718 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a02d3-6d93-43fc-874b-eef3bf0aff6a None admin] RBAC: Adding query filter params (interface=admin, service_id=7e537b1fb16646e695c4e97bac8b6b3a, region_id=RegionOne) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:31.869027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a02d3-6d93-43fc-874b-eef3bf0aff6a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:31.870465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a02d3-6d93-43fc-874b-eef3bf0aff6a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:31.876519 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 489/975] 10.222.0.78 () {56 vars in 1249 bytes} [Wed May 25 14:42:31 2022] GET /identity/v3/endpoints?service_id=7e537b1fb16646e695c4e97bac8b6b3a&interface=admin®ion_id=RegionOne => generated 193 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:31.880428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-f84d7235-13cb-4634-b314-4a9bf6f88ae3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:31.880428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:31.880428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:31.880428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:31.880428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:31.880428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:31.880428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:31.880428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:31.880428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:31.880428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:31.880428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:31.880428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:31.880428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:31.880428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:31.880428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:31.881791 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:31.881791 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:31.881791 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:31.881791 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:31.881791 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:31.881791 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:31.881791 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:31.881791 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:31.881791 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:31.881791 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:31.881791 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:31.881791 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:31.881791 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:31.881791 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:31.881791 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:31.881791 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:31.881791 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:31.883467 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:31.883467 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:31.883467 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:31.883467 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:31.883467 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 487/976] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:31.885515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a02d3-6d93-43fc-874b-eef3bf0aff6a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:31.887734 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d7235-13cb-4634-b314-4a9bf6f88ae3 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:32.004702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:32.035291 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d7235-13cb-4634-b314-4a9bf6f88ae3 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:32.035898 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a02d3-6d93-43fc-874b-eef3bf0aff6a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:32.079433 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5608a-daa6-4cd3-8a5f-6dc2230b7ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:32.080103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5608a-daa6-4cd3-8a5f-6dc2230b7ba6 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:32.080382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5608a-daa6-4cd3-8a5f-6dc2230b7ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:32.080659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5608a-daa6-4cd3-8a5f-6dc2230b7ba6 None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:32.081400 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5608a-daa6-4cd3-8a5f-6dc2230b7ba6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:32.081705 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5608a-daa6-4cd3-8a5f-6dc2230b7ba6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:32.083136 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5608a-daa6-4cd3-8a5f-6dc2230b7ba6 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:32.104508 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 488/977] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:32.104768 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30758130-27b0-4f3a-9b16-13371c892204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:32.104768 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30758130-27b0-4f3a-9b16-13371c892204 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:32.104768 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30758130-27b0-4f3a-9b16-13371c892204 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:32.104768 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30758130-27b0-4f3a-9b16-13371c892204 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:32.105414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30758130-27b0-4f3a-9b16-13371c892204 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:32.105414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30758130-27b0-4f3a-9b16-13371c892204 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:32.107292 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5608a-daa6-4cd3-8a5f-6dc2230b7ba6 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:32.113843 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 490/978] 10.222.0.78 () {56 vars in 1087 bytes} [Wed May 25 14:42:31 2022] GET /identity/v3/services => generated 1497 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:42:32.213580 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23876fe9-9b40-4f5e-b029-2c934f5f5296 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:32.213948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23876fe9-9b40-4f5e-b029-2c934f5f5296 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:32.214444 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23876fe9-9b40-4f5e-b029-2c934f5f5296 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:32.217001 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 491/979] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:32.225425 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011444a3-4e76-4195-b4f6-5046123f3d63 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:32.225886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011444a3-4e76-4195-b4f6-5046123f3d63 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:32.226347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011444a3-4e76-4195-b4f6-5046123f3d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:32.253432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-011444a3-4e76-4195-b4f6-5046123f3d63 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:32.255358 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-011444a3-4e76-4195-b4f6-5046123f3d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:32.262993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5608a-daa6-4cd3-8a5f-6dc2230b7ba6 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:32.282464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:32.316251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd53da-d1f4-46da-a18d-a3871e7886a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:32.317079 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd53da-d1f4-46da-a18d-a3871e7886a3 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:32.317352 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd53da-d1f4-46da-a18d-a3871e7886a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:32.317620 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd53da-d1f4-46da-a18d-a3871e7886a3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:32.331606 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cd53da-d1f4-46da-a18d-a3871e7886a3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:32.331606 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cd53da-d1f4-46da-a18d-a3871e7886a3 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:32.333773 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:32.350089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-27cd53da-d1f4-46da-a18d-a3871e7886a3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:42:32.350089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:32.350089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:32.350089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:32.350089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:32.350089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:32.350089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:32.350089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:32.350089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:32.350089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:32.350089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:32.350089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:32.350089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:42:32.350089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:42:32.350089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:42:32.350089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:42:32.351103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:32.351103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:32.351103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:42:32.351103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:32.351103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:42:32.351103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:32.351103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:32.351103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:32.351103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:32.351103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:32.351103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:32.351103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:32.351103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:32.351103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:32.351103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:32.351103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:32.351103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:32.351103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:32.353294 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:32.353294 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:32.353294 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:42:32.353294 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:42:32.353294 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:42:32.353294 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:42:32.353294 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:42:32.353294 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:42:32.353294 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:42:32.353294 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:32.353294 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 489/980] 10.222.0.78 () {56 vars in 1117 bytes} [Wed May 25 14:42:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:32.358551 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd53da-d1f4-46da-a18d-a3871e7886a3 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:32.373514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 492/981] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:32.397190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011444a3-4e76-4195-b4f6-5046123f3d63 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:32.494045 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd53da-d1f4-46da-a18d-a3871e7886a3 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:32.534738 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7269206-aa5f-4bdc-a3b6-54d0245df659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:32.535232 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7269206-aa5f-4bdc-a3b6-54d0245df659 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:32.535515 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7269206-aa5f-4bdc-a3b6-54d0245df659 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:32.535788 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7269206-aa5f-4bdc-a3b6-54d0245df659 None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:32.536944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7269206-aa5f-4bdc-a3b6-54d0245df659 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:32.537072 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7269206-aa5f-4bdc-a3b6-54d0245df659 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:32.538485 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7269206-aa5f-4bdc-a3b6-54d0245df659 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:32.542003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011444a3-4e76-4195-b4f6-5046123f3d63 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:32.558470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 490/982] 10.222.0.78 () {56 vars in 1149 bytes} [Wed May 25 14:42:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:32.564364 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7269206-aa5f-4bdc-a3b6-54d0245df659 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:32.593434 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152ed81-efb9-4ebe-aa8b-a717e8358d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:32.594148 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152ed81-efb9-4ebe-aa8b-a717e8358d0c None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:32.594440 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152ed81-efb9-4ebe-aa8b-a717e8358d0c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:32.595167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152ed81-efb9-4ebe-aa8b-a717e8358d0c None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:32.595466 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f152ed81-efb9-4ebe-aa8b-a717e8358d0c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:32.596880 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f152ed81-efb9-4ebe-aa8b-a717e8358d0c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:32.613463 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 493/983] 10.222.0.78 () {60 vars in 1141 bytes} [Wed May 25 14:42:32 2022] POST /identity/v3/services => generated 242 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:42:32.654064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:32.711155 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7269206-aa5f-4bdc-a3b6-54d0245df659 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:32.762390 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619c5a7-b938-484a-8a89-5a6c8fe90002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:32.763062 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619c5a7-b938-484a-8a89-5a6c8fe90002 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:32.763408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619c5a7-b938-484a-8a89-5a6c8fe90002 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:32.763680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619c5a7-b938-484a-8a89-5a6c8fe90002 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:32.854705 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a619c5a7-b938-484a-8a89-5a6c8fe90002 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:32.856229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a619c5a7-b938-484a-8a89-5a6c8fe90002 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:32.919290 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:32.986083 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 491/984] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:42:32 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 14:42:32.991275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152ed81-efb9-4ebe-aa8b-a717e8358d0c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:33.013507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:33.174300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152ed81-efb9-4ebe-aa8b-a717e8358d0c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:33.224654 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded4ee3-cdcf-4365-a0c0-084357d1ab41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:33.225876 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded4ee3-cdcf-4365-a0c0-084357d1ab41 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:33.226429 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded4ee3-cdcf-4365-a0c0-084357d1ab41 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:33.226931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded4ee3-cdcf-4365-a0c0-084357d1ab41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:33.240913 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded4ee3-cdcf-4365-a0c0-084357d1ab41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:33.242847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded4ee3-cdcf-4365-a0c0-084357d1ab41 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:33.263711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-bded4ee3-cdcf-4365-a0c0-084357d1ab41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:33.263711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:33.263711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:33.263711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:33.263711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:33.263711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:33.263711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:33.263711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:33.263711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:33.263711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:33.263711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:33.263711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:33.263711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:33.263711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:33.263711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:33.264667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:33.264667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:33.264667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:33.264667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:33.264667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:33.264667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:33.264667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:33.264667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:33.264667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:33.264667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:33.264667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:33.264667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:33.264667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:33.264667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:33.264667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:33.264667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:33.264667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:33.265750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:33.265750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:33.265750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:33.265750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:33.272221 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 494/985] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:33.273847 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619c5a7-b938-484a-8a89-5a6c8fe90002 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:33.420882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619c5a7-b938-484a-8a89-5a6c8fe90002 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:33.460125 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e3455-aff7-4b23-a96b-5191cf95b619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:33.460933 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e3455-aff7-4b23-a96b-5191cf95b619 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:33.461214 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e3455-aff7-4b23-a96b-5191cf95b619 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:33.461483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e3455-aff7-4b23-a96b-5191cf95b619 None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:33.462250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e3455-aff7-4b23-a96b-5191cf95b619 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:33.462551 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e3455-aff7-4b23-a96b-5191cf95b619 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:33.463925 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e3455-aff7-4b23-a96b-5191cf95b619 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:33.482117 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 492/986] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:33.483832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded4ee3-cdcf-4365-a0c0-084357d1ab41 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:33.553119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:33.598287 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded4ee3-cdcf-4365-a0c0-084357d1ab41 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:33.641052 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3ba09-772f-4c31-a57e-787af410f87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:33.641859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3ba09-772f-4c31-a57e-787af410f87e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:33.642172 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3ba09-772f-4c31-a57e-787af410f87e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:33.642451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3ba09-772f-4c31-a57e-787af410f87e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:33.649698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e3ba09-772f-4c31-a57e-787af410f87e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:33.651225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e3ba09-772f-4c31-a57e-787af410f87e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:33.667341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-e5e3ba09-772f-4c31-a57e-787af410f87e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:33.667341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:33.667341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:33.667341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:33.667341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:33.667341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:33.667341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:33.667341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:33.667341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:33.667341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:33.667341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:33.667341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:33.667341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:42:33.667341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:42:33.667341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:42:33.668362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:42:33.668362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:33.668362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:33.668362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:33.668362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:33.668362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:33.668362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:33.668362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:33.668362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:33.668362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:33.668362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:33.668362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:33.668362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:33.668362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:33.668362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:33.668362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:33.668362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:33.669328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:42:33.669328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:42:33.669328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:42:33.669328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:42:33.669328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:42:33.669328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:42:33.669328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:33.669328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:33.673735 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 495/987] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:42:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:33.674433 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e3455-aff7-4b23-a96b-5191cf95b619 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:33.692935 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:33.806907 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e3455-aff7-4b23-a96b-5191cf95b619 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:33.846898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a2e6c-90e7-433b-8477-c82f26d1b7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:33.847560 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a2e6c-90e7-433b-8477-c82f26d1b7c1 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:33.847837 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a2e6c-90e7-433b-8477-c82f26d1b7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:33.848105 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a2e6c-90e7-433b-8477-c82f26d1b7c1 None admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:33.848890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36a2e6c-90e7-433b-8477-c82f26d1b7c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:33.849189 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36a2e6c-90e7-433b-8477-c82f26d1b7c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:33.850671 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36a2e6c-90e7-433b-8477-c82f26d1b7c1 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:33.864847 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 493/988] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:42:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:33.872075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3ba09-772f-4c31-a57e-787af410f87e None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:34.009771 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3ba09-772f-4c31-a57e-787af410f87e None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:34.035140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:34.048914 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69d616-fd31-44e3-8997-a3c33c243ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:34.049583 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69d616-fd31-44e3-8997-a3c33c243ba7 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:34.049868 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69d616-fd31-44e3-8997-a3c33c243ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:34.050177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69d616-fd31-44e3-8997-a3c33c243ba7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:34.051020 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a69d616-fd31-44e3-8997-a3c33c243ba7 None admin] RBAC: Adding query filter params (role.id=074ec0dc23ca41fbb4833f8857859512, scope.project.id=908a13d2b49544fbaf760b07e213bb7d, user.id=b2935579c8554bed8b1e211c47548ac5) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:34.051337 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a69d616-fd31-44e3-8997-a3c33c243ba7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:34.052824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a69d616-fd31-44e3-8997-a3c33c243ba7 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:34.070168 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 496/989] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:42:33 2022] GET /identity/v3/role_assignments?user.id=b2935579c8554bed8b1e211c47548ac5&scope.project.id=908a13d2b49544fbaf760b07e213bb7d&role.id=074ec0dc23ca41fbb4833f8857859512 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:34.481689 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244983bb-3d62-47b0-9903-9678f1f4de42 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:34.482247 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244983bb-3d62-47b0-9903-9678f1f4de42 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:34.482247 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244983bb-3d62-47b0-9903-9678f1f4de42 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:34.483645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 494/990] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:34.489665 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8d66c-4308-45d9-9277-09ef776ebd8a None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:34.490175 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8d66c-4308-45d9-9277-09ef776ebd8a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:34.490284 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8d66c-4308-45d9-9277-09ef776ebd8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:34.513251 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-49f8d66c-4308-45d9-9277-09ef776ebd8a None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:34.515365 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-49f8d66c-4308-45d9-9277-09ef776ebd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:34.605973 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 497/991] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:42:34.629833 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244983bb-3d62-47b0-9903-9678f1f4de42 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:34.706628 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:34.743201 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244983bb-3d62-47b0-9903-9678f1f4de42 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:34.785888 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b21767-11f6-44df-8f7b-753f4eb3dfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:34.786756 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b21767-11f6-44df-8f7b-753f4eb3dfae None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:34.787038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b21767-11f6-44df-8f7b-753f4eb3dfae None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:34.787312 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b21767-11f6-44df-8f7b-753f4eb3dfae None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:34.787988 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b21767-11f6-44df-8f7b-753f4eb3dfae None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:34.789440 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b21767-11f6-44df-8f7b-753f4eb3dfae None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:34.803140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-47b21767-11f6-44df-8f7b-753f4eb3dfae None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:42:34.803140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:34.803140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:42:34.803140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:42:34.803140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:42:34.803140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:42:34.803140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:42:34.803140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:34.803140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:42:34.803140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:34.803140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:42:34.803140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:34.803140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:34.803140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:34.803140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:34.803140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:34.803140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:34.804252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:34.804252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:34.804252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:34.804252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:34.804252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:34.804252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:34.804252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:42:34.804252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:42:34.804252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:42:34.804252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:42:34.804252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:34.804252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:34.804252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:34.804252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:34.804252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:34.804252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:34.811015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:34.811015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:34.811015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:34.811015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:34.811015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:34.811015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:34.811015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:34.811015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:34.811015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:34.811015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:34.811015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:42:34.811015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:34.811015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:42:34.811015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:34.811015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 495/992] 10.222.0.78 () {56 vars in 1126 bytes} [Wed May 25 14:42:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:34.813374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f8d66c-4308-45d9-9277-09ef776ebd8a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:34.939606 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f8d66c-4308-45d9-9277-09ef776ebd8a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:34.958313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:34.961697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1a6f9-1ebc-4c8e-ad4d-ba4207110840 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:34.961998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1a6f9-1ebc-4c8e-ad4d-ba4207110840 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:34.962332 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1a6f9-1ebc-4c8e-ad4d-ba4207110840 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:34.963971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 496/993] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:34.971498 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e153d-dcf1-441f-a76e-6ad240c02cf5 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:34.971784 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e153d-dcf1-441f-a76e-6ad240c02cf5 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:34.972055 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e153d-dcf1-441f-a76e-6ad240c02cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:34.984871 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd882e1-5486-4e8b-ad57-63b1b6e4d8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:34.986098 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd882e1-5486-4e8b-ad57-63b1b6e4d8ff None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:34.986386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd882e1-5486-4e8b-ad57-63b1b6e4d8ff None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:34.986829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd882e1-5486-4e8b-ad57-63b1b6e4d8ff None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:34.987544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd882e1-5486-4e8b-ad57-63b1b6e4d8ff None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:34.987860 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd882e1-5486-4e8b-ad57-63b1b6e4d8ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:34.989245 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd882e1-5486-4e8b-ad57-63b1b6e4d8ff None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:34.990822 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-e11e153d-dcf1-441f-a76e-6ad240c02cf5 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:34.992011 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-e11e153d-dcf1-441f-a76e-6ad240c02cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:34.994837 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 498/994] 10.222.0.78 () {56 vars in 1122 bytes} [Wed May 25 14:42:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:35.000562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd882e1-5486-4e8b-ad57-63b1b6e4d8ff None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:35.077196 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 497/995] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:42:35.099697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e153d-dcf1-441f-a76e-6ad240c02cf5 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:35.116980 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd882e1-5486-4e8b-ad57-63b1b6e4d8ff None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:35.154572 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385dee7-38d5-4337-9fcd-02d391cb5dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:35.155373 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385dee7-38d5-4337-9fcd-02d391cb5dd1 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:35.155550 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385dee7-38d5-4337-9fcd-02d391cb5dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:35.155861 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385dee7-38d5-4337-9fcd-02d391cb5dd1 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:35.156554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385dee7-38d5-4337-9fcd-02d391cb5dd1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:35.156866 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385dee7-38d5-4337-9fcd-02d391cb5dd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:35.158301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385dee7-38d5-4337-9fcd-02d391cb5dd1 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:35.164544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 499/996] 10.222.0.78 () {56 vars in 1122 bytes} [Wed May 25 14:42:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:35.170733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385dee7-38d5-4337-9fcd-02d391cb5dd1 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:35.181517 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:35.226326 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e153d-dcf1-441f-a76e-6ad240c02cf5 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:35.229751 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:35.270223 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e857526-6dd4-4aec-b640-e468d8cebae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:35.270951 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e857526-6dd4-4aec-b640-e468d8cebae9 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:35.271232 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e857526-6dd4-4aec-b640-e468d8cebae9 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:35.271906 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e857526-6dd4-4aec-b640-e468d8cebae9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:35.272162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e857526-6dd4-4aec-b640-e468d8cebae9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:35.273549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e857526-6dd4-4aec-b640-e468d8cebae9 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:35.286774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-4e857526-6dd4-4aec-b640-e468d8cebae9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:42:35.286774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:35.286774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:42:35.286774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:42:35.286774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:42:35.286774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:42:35.286774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:42:35.286774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:35.286774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:42:35.286774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:35.286774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:42:35.286774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:35.286774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:35.286774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:35.286774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:35.286774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:35.286774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:35.287858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:35.287858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:35.287858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:35.287858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:35.287858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:35.287858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:35.287858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:42:35.287858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:42:35.287858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:42:35.287858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:42:35.287858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:35.287858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:35.287858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:35.287858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:35.287858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:35.287858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:35.288936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:35.288936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:35.288936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:35.288936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:35.288936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:35.288936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:35.288936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:35.288936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:35.288936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:35.288936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:35.288936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:42:35.288936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:35.288936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:42:35.288936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:35.288936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 498/997] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:42:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:35.298876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e857526-6dd4-4aec-b640-e468d8cebae9 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:35.302597 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385dee7-38d5-4337-9fcd-02d391cb5dd1 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:35.343786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579d199-052c-45a5-ab02-f9f32d19e525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:35.344576 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579d199-052c-45a5-ab02-f9f32d19e525 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:35.344865 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579d199-052c-45a5-ab02-f9f32d19e525 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:35.345153 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579d199-052c-45a5-ab02-f9f32d19e525 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:35.346145 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3579d199-052c-45a5-ab02-f9f32d19e525 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:35.347371 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3579d199-052c-45a5-ab02-f9f32d19e525 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:35.374515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 500/998] 10.222.0.78 () {60 vars in 1144 bytes} [Wed May 25 14:42:35 2022] POST /identity/v3/endpoints => generated 325 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:42:35.430446 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e857526-6dd4-4aec-b640-e468d8cebae9 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:35.463918 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:35.477528 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486ec02-ede1-4326-afd5-03baa83ac53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:35.478158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486ec02-ede1-4326-afd5-03baa83ac53e None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:35.478444 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486ec02-ede1-4326-afd5-03baa83ac53e None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:35.478729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486ec02-ede1-4326-afd5-03baa83ac53e None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:35.479462 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486ec02-ede1-4326-afd5-03baa83ac53e None admin] RBAC: Adding query filter params (name=compute) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:35.479771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486ec02-ede1-4326-afd5-03baa83ac53e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:35.481219 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486ec02-ede1-4326-afd5-03baa83ac53e None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:35.492037 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 499/999] 10.222.0.78 () {56 vars in 1112 bytes} [Wed May 25 14:42:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:35.494253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579d199-052c-45a5-ab02-f9f32d19e525 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:35.610962 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579d199-052c-45a5-ab02-f9f32d19e525 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:35.613398 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:35.648378 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263af8d-3aa3-48c6-b951-e4ae0f486eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:35.649038 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263af8d-3aa3-48c6-b951-e4ae0f486eab None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:35.649319 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263af8d-3aa3-48c6-b951-e4ae0f486eab None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:35.649597 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263af8d-3aa3-48c6-b951-e4ae0f486eab None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:35.650327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c263af8d-3aa3-48c6-b951-e4ae0f486eab None admin] RBAC: Adding query filter params (type=compute) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:35.650633 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c263af8d-3aa3-48c6-b951-e4ae0f486eab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:35.652077 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c263af8d-3aa3-48c6-b951-e4ae0f486eab None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:35.657230 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 501/1000] 10.222.0.78 () {56 vars in 1112 bytes} [Wed May 25 14:42:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:35.663642 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486ec02-ede1-4326-afd5-03baa83ac53e None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:35.768340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486ec02-ede1-4326-afd5-03baa83ac53e None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:35.804706 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c447d-b334-48b0-9fd2-17c45d7c7f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:35.805367 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c447d-b334-48b0-9fd2-17c45d7c7f6b None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:35.805647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c447d-b334-48b0-9fd2-17c45d7c7f6b None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:35.805915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c447d-b334-48b0-9fd2-17c45d7c7f6b None admin] PATH_INFO: `/v3/endpoints` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:35.806740 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c447d-b334-48b0-9fd2-17c45d7c7f6b None admin] RBAC: Adding query filter params (interface=public, service_id=73bbee7763704dcbaf46f9643a3bee3e, region_id=RegionOne) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:35.807049 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c447d-b334-48b0-9fd2-17c45d7c7f6b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:35.808977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c447d-b334-48b0-9fd2-17c45d7c7f6b None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:35.819075 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 500/1001] 10.222.0.78 () {56 vars in 1251 bytes} [Wed May 25 14:42:35 2022] GET /identity/v3/endpoints?service_id=73bbee7763704dcbaf46f9643a3bee3e&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:35.820379 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263af8d-3aa3-48c6-b951-e4ae0f486eab None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:35.937732 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263af8d-3aa3-48c6-b951-e4ae0f486eab None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:35.974693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06014301-1c98-48c9-a09c-fb5afc13eea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:35.975356 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06014301-1c98-48c9-a09c-fb5afc13eea5 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:35.975638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06014301-1c98-48c9-a09c-fb5afc13eea5 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:35.975910 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06014301-1c98-48c9-a09c-fb5afc13eea5 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:35.976586 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06014301-1c98-48c9-a09c-fb5afc13eea5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:35.977966 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06014301-1c98-48c9-a09c-fb5afc13eea5 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:35.990302 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 502/1002] 10.222.0.78 () {56 vars in 1087 bytes} [Wed May 25 14:42:35 2022] GET /identity/v3/services => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:42:36.423635 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64aa65-f14e-4526-a864-e15ad1588b37 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:36.424035 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64aa65-f14e-4526-a864-e15ad1588b37 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:36.424277 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64aa65-f14e-4526-a864-e15ad1588b37 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:36.426817 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 501/1003] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:36.434819 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dff33-6269-4cc7-bd8f-1fc276e74497 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:36.435179 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dff33-6269-4cc7-bd8f-1fc276e74497 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:36.435644 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dff33-6269-4cc7-bd8f-1fc276e74497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:36.454210 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-fa2dff33-6269-4cc7-bd8f-1fc276e74497 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:36.455419 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-fa2dff33-6269-4cc7-bd8f-1fc276e74497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:36.484555 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:36.538590 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 503/1004] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:36 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:36.561920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64aa65-f14e-4526-a864-e15ad1588b37 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:36.633015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:36.682174 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64aa65-f14e-4526-a864-e15ad1588b37 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:36.717030 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101539c-1307-4b6c-ba61-d51aa05f6202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:36.717822 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101539c-1307-4b6c-ba61-d51aa05f6202 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:36.718123 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101539c-1307-4b6c-ba61-d51aa05f6202 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:36.718395 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101539c-1307-4b6c-ba61-d51aa05f6202 None admin] PATH_INFO: `/v3/services/image` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:36.719044 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6101539c-1307-4b6c-ba61-d51aa05f6202 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:36.720398 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6101539c-1307-4b6c-ba61-d51aa05f6202 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:36.733042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-6101539c-1307-4b6c-ba61-d51aa05f6202 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:42:36.733042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:36.733042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:42:36.733042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:42:36.733042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:42:36.733042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:42:36.733042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:42:36.733042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:36.733042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:42:36.733042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:36.733042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:42:36.733042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:36.733042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:36.733042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:36.733042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:36.733042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:36.733042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:36.734258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:36.734258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:36.734258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:36.734258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:36.734258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:36.734258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:36.734258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:42:36.734258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:42:36.734258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:42:36.734258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:42:36.734258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:36.734258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:36.734258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:36.734258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:36.734258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:36.734258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:36.735114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:36.735114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:36.735114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:36.735114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:36.735114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:36.735114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:36.735114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:36.735114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:36.735114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:36.735114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:36.735114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:42:36.735114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:36.735114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:42:36.735114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:36.735114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 502/1005] 10.222.0.78 () {56 vars in 1105 bytes} [Wed May 25 14:42:36 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:36.742902 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dff33-6269-4cc7-bd8f-1fc276e74497 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:36.863433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dff33-6269-4cc7-bd8f-1fc276e74497 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:36.922338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d94cca-c94a-4da7-9936-c6e017c03e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:36.923031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d94cca-c94a-4da7-9936-c6e017c03e8f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:36.923313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d94cca-c94a-4da7-9936-c6e017c03e8f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:36.923594 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d94cca-c94a-4da7-9936-c6e017c03e8f None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:36.924318 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d94cca-c94a-4da7-9936-c6e017c03e8f None admin] RBAC: Adding query filter params (name=image) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:36.924631 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d94cca-c94a-4da7-9936-c6e017c03e8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:36.926054 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d94cca-c94a-4da7-9936-c6e017c03e8f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:36.936457 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 504/1006] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:36.937917 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101539c-1307-4b6c-ba61-d51aa05f6202 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:37.051680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101539c-1307-4b6c-ba61-d51aa05f6202 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:37.071406 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:37.096139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdbda4e-399d-49a2-aad1-6e8979b3cfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:37.097169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbda4e-399d-49a2-aad1-6e8979b3cfd2 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:37.097525 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbda4e-399d-49a2-aad1-6e8979b3cfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:37.097808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbda4e-399d-49a2-aad1-6e8979b3cfd2 None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:37.098563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdbda4e-399d-49a2-aad1-6e8979b3cfd2 None admin] RBAC: Adding query filter params (type=image) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:37.098869 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdbda4e-399d-49a2-aad1-6e8979b3cfd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:37.100345 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdbda4e-399d-49a2-aad1-6e8979b3cfd2 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:37.114045 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 503/1007] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:36 2022] GET /identity/v3/services?type=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:37.746187 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a27967-2b5e-4d81-a21b-cf0f81f49b7d None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:37.746730 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a27967-2b5e-4d81-a21b-cf0f81f49b7d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:37.746730 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a27967-2b5e-4d81-a21b-cf0f81f49b7d None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:37.748298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 505/1008] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:42:37.753766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a75d4-89c1-4b5e-9142-1a14391eb0c2 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:37.754268 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a75d4-89c1-4b5e-9142-1a14391eb0c2 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:37.754348 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a75d4-89c1-4b5e-9142-1a14391eb0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:37.781737 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-e72a75d4-89c1-4b5e-9142-1a14391eb0c2 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:37.783768 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-e72a75d4-89c1-4b5e-9142-1a14391eb0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:37.866077 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 504/1009] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:37 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:42:37.890147 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a27967-2b5e-4d81-a21b-cf0f81f49b7d None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:37.941061 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:38.084266 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a27967-2b5e-4d81-a21b-cf0f81f49b7d None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:38.147522 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be8fdd-6a0e-4f55-9bf8-331b10e161c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:38.148889 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be8fdd-6a0e-4f55-9bf8-331b10e161c5 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:38.149383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be8fdd-6a0e-4f55-9bf8-331b10e161c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:38.149878 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be8fdd-6a0e-4f55-9bf8-331b10e161c5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:38.152641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be8fdd-6a0e-4f55-9bf8-331b10e161c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:38.155409 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be8fdd-6a0e-4f55-9bf8-331b10e161c5 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:38.164178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 506/1010] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:38.165539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a75d4-89c1-4b5e-9142-1a14391eb0c2 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:38.348940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a75d4-89c1-4b5e-9142-1a14391eb0c2 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:38.366158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:38.393809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc208d65-8e8d-469c-8419-f3cb7cd35c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:38.394524 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc208d65-8e8d-469c-8419-f3cb7cd35c6c None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:38.394801 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc208d65-8e8d-469c-8419-f3cb7cd35c6c None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:38.395074 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc208d65-8e8d-469c-8419-f3cb7cd35c6c None admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:38.395871 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc208d65-8e8d-469c-8419-f3cb7cd35c6c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:38.397358 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc208d65-8e8d-469c-8419-f3cb7cd35c6c None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:38.428338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 505/1011] 10.222.0.78 () {60 vars in 1141 bytes} [Wed May 25 14:42:38 2022] POST /identity/v3/projects => generated 311 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:42:38.441391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160bee0a-39d2-4465-a0fa-0f9d990a1a22 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:38.441686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160bee0a-39d2-4465-a0fa-0f9d990a1a22 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:38.441960 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160bee0a-39d2-4465-a0fa-0f9d990a1a22 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:38.443770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 507/1012] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:42:38.452168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f2517-ebf2-42b8-a7c2-7e1fe062279c None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:38.452168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f2517-ebf2-42b8-a7c2-7e1fe062279c None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:38.452168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f2517-ebf2-42b8-a7c2-7e1fe062279c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:38.471249 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-362f2517-ebf2-42b8-a7c2-7e1fe062279c None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:38.472508 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-362f2517-ebf2-42b8-a7c2-7e1fe062279c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:38.602952 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 506/1013] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:38 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:38.644099 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160bee0a-39d2-4465-a0fa-0f9d990a1a22 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:38.794868 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:38.805624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160bee0a-39d2-4465-a0fa-0f9d990a1a22 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:38.849227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f7969-fbc5-4a25-a0bc-931492b8fd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:38.850060 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f7969-fbc5-4a25-a0bc-931492b8fd66 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:38.850354 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f7969-fbc5-4a25-a0bc-931492b8fd66 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:38.850625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f7969-fbc5-4a25-a0bc-931492b8fd66 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:38.851287 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2f7969-fbc5-4a25-a0bc-931492b8fd66 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:38.852726 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2f7969-fbc5-4a25-a0bc-931492b8fd66 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:38.865986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-ad2f7969-fbc5-4a25-a0bc-931492b8fd66 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:42:38.865986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:38.865986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:42:38.865986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:42:38.865986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:42:38.865986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:42:38.865986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:42:38.865986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:38.865986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:42:38.865986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:38.865986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:42:38.865986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:38.865986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:38.865986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:38.865986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:38.865986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:38.865986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:38.868027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:38.868027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:38.868027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:38.868027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:38.868027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:38.868027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:38.868027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:42:38.868027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:42:38.868027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:42:38.868027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:42:38.868027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:38.868027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:38.868027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:38.868027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:38.868027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:38.868027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:38.869742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:38.869742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:38.869742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:38.869742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:38.869742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:38.869742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:38.869742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:38.869742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:38.869742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:38.869742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:38.869742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:42:38.869742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:38.869742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:42:38.869742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:38.869742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 508/1014] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:42:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:38.875759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f2517-ebf2-42b8-a7c2-7e1fe062279c None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:39.059025 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:39.069128 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f2517-ebf2-42b8-a7c2-7e1fe062279c None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:39.112410 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de1087-23d2-4fc2-865d-39e6a2caf00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:39.113090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de1087-23d2-4fc2-865d-39e6a2caf00b None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:39.113370 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de1087-23d2-4fc2-865d-39e6a2caf00b None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:39.113646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de1087-23d2-4fc2-865d-39e6a2caf00b None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:39.114402 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de1087-23d2-4fc2-865d-39e6a2caf00b None admin] RBAC: Adding query filter params (name=compute) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:39.115075 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de1087-23d2-4fc2-865d-39e6a2caf00b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:39.116188 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de1087-23d2-4fc2-865d-39e6a2caf00b None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:39.123020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 507/1015] 10.222.0.78 () {56 vars in 1112 bytes} [Wed May 25 14:42:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:39.129362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f7969-fbc5-4a25-a0bc-931492b8fd66 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:39.289243 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f7969-fbc5-4a25-a0bc-931492b8fd66 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:39.355890 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a71e44-15e0-42bc-a52a-bdeca2151e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:39.357049 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a71e44-15e0-42bc-a52a-bdeca2151e85 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:39.357552 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a71e44-15e0-42bc-a52a-bdeca2151e85 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:39.358075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a71e44-15e0-42bc-a52a-bdeca2151e85 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:39.359302 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a71e44-15e0-42bc-a52a-bdeca2151e85 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:39.359855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a71e44-15e0-42bc-a52a-bdeca2151e85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:39.362604 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a71e44-15e0-42bc-a52a-bdeca2151e85 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:39.378375 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 509/1016] 10.222.0.78 () {56 vars in 1112 bytes} [Wed May 25 14:42:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:39.382491 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de1087-23d2-4fc2-865d-39e6a2caf00b None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:39.483956 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429097da-72ff-4a67-ba5f-a30f8fbfe52f None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:39.484392 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429097da-72ff-4a67-ba5f-a30f8fbfe52f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:39.484545 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429097da-72ff-4a67-ba5f-a30f8fbfe52f None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:39.485900 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 510/1017] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:42:39.491489 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29994115-b8a7-4c27-b382-9e84af64ea6f None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:39.491880 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29994115-b8a7-4c27-b382-9e84af64ea6f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:39.492048 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29994115-b8a7-4c27-b382-9e84af64ea6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:39.512634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-29994115-b8a7-4c27-b382-9e84af64ea6f None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:39.513840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-29994115-b8a7-4c27-b382-9e84af64ea6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:39.534538 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:39.574890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de1087-23d2-4fc2-865d-39e6a2caf00b None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:39.619421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 511/1018] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:39 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:39.643142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dbdaa0-b7b3-4439-9ee8-7f3a3a2b93d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:39.644232 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbdaa0-b7b3-4439-9ee8-7f3a3a2b93d7 None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:39.644719 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbdaa0-b7b3-4439-9ee8-7f3a3a2b93d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:39.645204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbdaa0-b7b3-4439-9ee8-7f3a3a2b93d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:39.646661 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dbdaa0-b7b3-4439-9ee8-7f3a3a2b93d7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:39.648681 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dbdaa0-b7b3-4439-9ee8-7f3a3a2b93d7 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:39.660114 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29994115-b8a7-4c27-b382-9e84af64ea6f None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:39.681518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 508/1019] 10.222.0.78 () {60 vars in 1144 bytes} [Wed May 25 14:42:39 2022] POST /identity/v3/endpoints => generated 334 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:42:39.846964 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29994115-b8a7-4c27-b382-9e84af64ea6f None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:39.919923 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e04714-0d16-42f4-ae49-ffbe95b4e72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:39.920678 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e04714-0d16-42f4-ae49-ffbe95b4e72a None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:39.920969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e04714-0d16-42f4-ae49-ffbe95b4e72a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:39.921241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e04714-0d16-42f4-ae49-ffbe95b4e72a None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:39.923581 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e04714-0d16-42f4-ae49-ffbe95b4e72a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:39.923581 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e04714-0d16-42f4-ae49-ffbe95b4e72a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:39.942521 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 512/1020] 10.222.0.78 () {60 vars in 1141 bytes} [Wed May 25 14:42:39 2022] POST /identity/v3/services => generated 242 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:42:41.367625 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a07e6-bb64-451b-b328-b12fd9aed309 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:41.368428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a07e6-bb64-451b-b328-b12fd9aed309 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:41.368428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a07e6-bb64-451b-b328-b12fd9aed309 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:41.369838 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 509/1021] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:41.375566 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9cd9f-a7c4-41f3-a4d3-96d1f701e176 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:41.376058 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9cd9f-a7c4-41f3-a4d3-96d1f701e176 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:41.376146 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9cd9f-a7c4-41f3-a4d3-96d1f701e176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:41.393769 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-eea9cd9f-a7c4-41f3-a4d3-96d1f701e176 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:41.395017 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-eea9cd9f-a7c4-41f3-a4d3-96d1f701e176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:41.397104 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:41.491598 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 513/1022] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:41.517872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a07e6-bb64-451b-b328-b12fd9aed309 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:41.549579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:41.723743 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a07e6-bb64-451b-b328-b12fd9aed309 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:41.790291 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4244f-5515-4c76-8d80-6542d5c3915e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:41.791675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4244f-5515-4c76-8d80-6542d5c3915e None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:41.792172 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4244f-5515-4c76-8d80-6542d5c3915e None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:41.792701 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4244f-5515-4c76-8d80-6542d5c3915e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:41.795037 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4244f-5515-4c76-8d80-6542d5c3915e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:41.797891 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4244f-5515-4c76-8d80-6542d5c3915e None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:41.805647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 510/1023] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:41.807094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9cd9f-a7c4-41f3-a4d3-96d1f701e176 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:41.960125 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9cd9f-a7c4-41f3-a4d3-96d1f701e176 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:42.028472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088afedd-df7d-4002-90e7-67d1ce16c3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:42.029633 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088afedd-df7d-4002-90e7-67d1ce16c3ea None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:42.030179 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088afedd-df7d-4002-90e7-67d1ce16c3ea None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:42.030704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088afedd-df7d-4002-90e7-67d1ce16c3ea None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:42.032153 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088afedd-df7d-4002-90e7-67d1ce16c3ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:42.035257 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088afedd-df7d-4002-90e7-67d1ce16c3ea None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:42.046749 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:42.064228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 514/1024] 10.222.0.78 () {60 vars in 1132 bytes} [Wed May 25 14:42:41 2022] POST /identity/v3/users => generated 285 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:42:42.185834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524049b-537f-4785-9a12-316d7eb40f71 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:42.186136 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524049b-537f-4785-9a12-316d7eb40f71 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:42.186478 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524049b-537f-4785-9a12-316d7eb40f71 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:42.188126 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 511/1025] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:42.193886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e660d-2381-48f9-9a9a-06d781ac7fbb None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:42.194219 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e660d-2381-48f9-9a9a-06d781ac7fbb None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:42.194689 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e660d-2381-48f9-9a9a-06d781ac7fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:42.213663 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-915e660d-2381-48f9-9a9a-06d781ac7fbb None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:42.215515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-915e660d-2381-48f9-9a9a-06d781ac7fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:42.315911 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 515/1026] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:42.341539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524049b-537f-4785-9a12-316d7eb40f71 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:42.458889 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:42.482433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2f048-9a93-414d-828e-1b8d84f47f7b None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:42.482960 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2f048-9a93-414d-828e-1b8d84f47f7b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:42.482960 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2f048-9a93-414d-828e-1b8d84f47f7b None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:42.484600 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 516/1027] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:42:42.485856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524049b-537f-4785-9a12-316d7eb40f71 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:42.491896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040f88a-8979-4af7-9909-003219b90acc None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:42.492309 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040f88a-8979-4af7-9909-003219b90acc None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:42.492567 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040f88a-8979-4af7-9909-003219b90acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:42.513447 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-0040f88a-8979-4af7-9909-003219b90acc None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:42.515000 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-0040f88a-8979-4af7-9909-003219b90acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:42.538312 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c180d4c-c2ff-494f-a9b0-d438b98ac0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:42.539702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c180d4c-c2ff-494f-a9b0-d438b98ac0db None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:42.540178 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c180d4c-c2ff-494f-a9b0-d438b98ac0db None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:42.540840 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c180d4c-c2ff-494f-a9b0-d438b98ac0db None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:42.563136 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c180d4c-c2ff-494f-a9b0-d438b98ac0db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:42:42.581650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-1c180d4c-c2ff-494f-a9b0-d438b98ac0db None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:42:42.581650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:42.581650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:42.581650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:42.581650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:42.581650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:42.581650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:42.581650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:42.581650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:42.581650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:42.581650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:42.581650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:42.581650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:42:42.581650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:42:42.581650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:42:42.581650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:42:42.582676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:42:42.582676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:42:42.582676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:42:42.582676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:42:42.582676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:42.582676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:42.582676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:42.582676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:42.582676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:42.582676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:42.582676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:42.582676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:42.582676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:42.582676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:42.582676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:42.582676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:42.583549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:42.583549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:42.583549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:42.583549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:42.583549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:42:42.583549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:42:42.583549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:42:42.583549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:42:42.583549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:42:42.583549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:42:42.583549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:42:42.583549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:42.584226 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 512/1028] 10.222.0.78 () {56 vars in 1120 bytes} [Wed May 25 14:42:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:42.590325 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c180d4c-c2ff-494f-a9b0-d438b98ac0db None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:42.606082 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 517/1029] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:42:42.629893 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040f88a-8979-4af7-9909-003219b90acc None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:42.699489 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:42.730128 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c180d4c-c2ff-494f-a9b0-d438b98ac0db None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:42.767818 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040f88a-8979-4af7-9909-003219b90acc None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:42.789027 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd91ef2-537d-442b-a400-5d51384a7fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:42.789708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd91ef2-537d-442b-a400-5d51384a7fd8 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:42.789989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd91ef2-537d-442b-a400-5d51384a7fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:42.790477 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd91ef2-537d-442b-a400-5d51384a7fd8 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:42.791057 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd91ef2-537d-442b-a400-5d51384a7fd8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:42.791361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd91ef2-537d-442b-a400-5d51384a7fd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:42.792778 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd91ef2-537d-442b-a400-5d51384a7fd8 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:42.803176 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 513/1030] 10.222.0.78 () {56 vars in 1115 bytes} [Wed May 25 14:42:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:42.810680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd91ef2-537d-442b-a400-5d51384a7fd8 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:42.815665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:42.822001 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a35f8b-abfb-4071-a492-da2fc6ba0060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:42.824262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a35f8b-abfb-4071-a492-da2fc6ba0060 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:42.824745 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a35f8b-abfb-4071-a492-da2fc6ba0060 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:42.825230 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a35f8b-abfb-4071-a492-da2fc6ba0060 None admin] PATH_INFO: `/v3/services/image` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:42.826477 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a35f8b-abfb-4071-a492-da2fc6ba0060 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:42.829001 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a35f8b-abfb-4071-a492-da2fc6ba0060 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:42.851391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-f7a35f8b-abfb-4071-a492-da2fc6ba0060 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:42:42.851391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:42.851391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:42:42.851391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:42:42.851391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:42:42.851391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:42:42.851391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:42:42.851391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:42.851391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:42:42.851391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:42.851391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:42:42.851391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:42.851391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:42.851391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:42.851391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:42.851391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:42.851391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:42.852432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:42.852432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:42.852432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:42.852432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:42.852432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:42.852432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:42.852432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:42:42.852432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:42:42.852432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:42:42.852432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:42:42.852432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:42.852432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:42.852432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:42.852432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:42.852432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:42.852432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:42.853322 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:42.853322 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:42.853322 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:42.853322 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:42.853322 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:42.853322 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:42.853322 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:42.853322 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:42.853322 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:42.853322 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:42.853322 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:42:42.853322 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:42.853322 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:42:42.853322 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:42.862384 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 518/1031] 10.222.0.78 () {56 vars in 1105 bytes} [Wed May 25 14:42:42 2022] GET /identity/v3/services/image => generated 86 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:42.864558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a35f8b-abfb-4071-a492-da2fc6ba0060 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:42.981441 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd91ef2-537d-442b-a400-5d51384a7fd8 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:43.019009 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a346403-a2d6-49cb-a4c7-2d68c0339b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:43.019836 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a346403-a2d6-49cb-a4c7-2d68c0339b21 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:43.020215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a346403-a2d6-49cb-a4c7-2d68c0339b21 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:43.020490 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a346403-a2d6-49cb-a4c7-2d68c0339b21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:43.033757 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a346403-a2d6-49cb-a4c7-2d68c0339b21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:43.035592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a346403-a2d6-49cb-a4c7-2d68c0339b21 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:43.036072 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a35f8b-abfb-4071-a492-da2fc6ba0060 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:43.055834 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:43.056340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-6a346403-a2d6-49cb-a4c7-2d68c0339b21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:43.056340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:43.056340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:43.056340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:43.056340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:43.056340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:43.056340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:43.056340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:43.056340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:43.056340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:43.056340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:43.056340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:43.056340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:43.056340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:43.056340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:43.057448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:43.057448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:43.057448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:43.057448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:43.057448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:43.057448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:43.057448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:43.057448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:43.057448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:43.057448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:43.057448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:43.057448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:43.057448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:43.057448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:43.057448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:43.057448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:43.057448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:43.058652 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:43.058652 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:43.058652 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:43.058652 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:43.058652 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 514/1032] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:43.065007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a346403-a2d6-49cb-a4c7-2d68c0339b21 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:43.084899 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082a0ca-81d9-4005-851d-5817ba93b097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:43.085758 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082a0ca-81d9-4005-851d-5817ba93b097 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:43.086148 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082a0ca-81d9-4005-851d-5817ba93b097 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:43.086432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082a0ca-81d9-4005-851d-5817ba93b097 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:43.087175 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7082a0ca-81d9-4005-851d-5817ba93b097 None admin] RBAC: Adding query filter params (name=image) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:43.087493 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7082a0ca-81d9-4005-851d-5817ba93b097 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:43.088928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7082a0ca-81d9-4005-851d-5817ba93b097 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:43.097243 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 519/1033] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:43.102070 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082a0ca-81d9-4005-851d-5817ba93b097 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:43.144030 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:43.204535 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a346403-a2d6-49cb-a4c7-2d68c0339b21 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:43.245662 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d1491-29ab-4268-b750-666999a9ab21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:43.246402 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d1491-29ab-4268-b750-666999a9ab21 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:43.246685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d1491-29ab-4268-b750-666999a9ab21 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:43.246964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d1491-29ab-4268-b750-666999a9ab21 None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:43.247708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d1491-29ab-4268-b750-666999a9ab21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:43.248026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d1491-29ab-4268-b750-666999a9ab21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:43.249435 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d1491-29ab-4268-b750-666999a9ab21 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:43.256994 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082a0ca-81d9-4005-851d-5817ba93b097 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:43.265287 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 515/1034] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:43.270900 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d1491-29ab-4268-b750-666999a9ab21 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:43.305534 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9ea17-2a9b-489f-a3df-5a29d45b5849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:43.306254 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9ea17-2a9b-489f-a3df-5a29d45b5849 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:43.306564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9ea17-2a9b-489f-a3df-5a29d45b5849 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:43.306845 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9ea17-2a9b-489f-a3df-5a29d45b5849 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:43.307566 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9ea17-2a9b-489f-a3df-5a29d45b5849 None admin] RBAC: Adding query filter params (type=image) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:43.308842 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9ea17-2a9b-489f-a3df-5a29d45b5849 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:43.309864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9ea17-2a9b-489f-a3df-5a29d45b5849 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:43.316514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 520/1035] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:43.322249 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9ea17-2a9b-489f-a3df-5a29d45b5849 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:43.377280 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:43.419958 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d1491-29ab-4268-b750-666999a9ab21 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:43.440433 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:43.456418 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9ea17-2a9b-489f-a3df-5a29d45b5849 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:43.459759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2163c-db68-46fc-8f11-f0264de2cbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:43.460567 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2163c-db68-46fc-8f11-f0264de2cbe8 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:43.460901 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2163c-db68-46fc-8f11-f0264de2cbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:43.461105 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2163c-db68-46fc-8f11-f0264de2cbe8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:43.469487 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a2163c-db68-46fc-8f11-f0264de2cbe8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:43.470981 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a2163c-db68-46fc-8f11-f0264de2cbe8 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:43.487585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-65a2163c-db68-46fc-8f11-f0264de2cbe8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:42:43.487585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:43.487585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:43.487585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:43.487585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:43.487585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:43.487585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:43.487585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:43.487585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:43.487585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:43.487585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:43.487585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:43.487585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:42:43.487585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:42:43.487585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:42:43.487585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:42:43.489520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:43.489520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:43.489520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:42:43.489520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:43.489520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:42:43.489520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:43.489520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:43.489520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:43.489520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:43.489520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:43.489520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:43.489520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:43.489520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:43.489520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:43.489520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:43.489520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:43.489520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:43.489520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:43.492760 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:43.492760 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:43.492760 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:42:43.492760 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:42:43.492760 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:42:43.492760 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:42:43.492760 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:42:43.492760 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:42:43.492760 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:42:43.492760 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:43.492760 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 516/1036] 10.222.0.78 () {56 vars in 1093 bytes} [Wed May 25 14:42:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:43.496184 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2163c-db68-46fc-8f11-f0264de2cbe8 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:43.496422 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb116c-52f4-4f5b-be80-0d51b3b91504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:43.497222 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb116c-52f4-4f5b-be80-0d51b3b91504 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:43.497222 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb116c-52f4-4f5b-be80-0d51b3b91504 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:43.497222 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb116c-52f4-4f5b-be80-0d51b3b91504 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:43.498378 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb116c-52f4-4f5b-be80-0d51b3b91504 None admin] RBAC: Adding query filter params (interface=public, service_id=048e354e674d4fc890846499f5e39e7d, region_id=RegionOne) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:43.499595 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb116c-52f4-4f5b-be80-0d51b3b91504 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:43.501024 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb116c-52f4-4f5b-be80-0d51b3b91504 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:43.508068 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 521/1037] 10.222.0.78 () {56 vars in 1251 bytes} [Wed May 25 14:42:43 2022] GET /identity/v3/endpoints?service_id=048e354e674d4fc890846499f5e39e7d&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:43.513723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb116c-52f4-4f5b-be80-0d51b3b91504 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:43.609602 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2163c-db68-46fc-8f11-f0264de2cbe8 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:43.652320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb116c-52f4-4f5b-be80-0d51b3b91504 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:43.653010 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a326450-d716-4679-a3e7-ef4da9284545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:43.654187 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a326450-d716-4679-a3e7-ef4da9284545 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:43.654731 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a326450-d716-4679-a3e7-ef4da9284545 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:43.655270 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a326450-d716-4679-a3e7-ef4da9284545 None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:43.656991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a326450-d716-4679-a3e7-ef4da9284545 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:43.657623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a326450-d716-4679-a3e7-ef4da9284545 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:43.660730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a326450-d716-4679-a3e7-ef4da9284545 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:43.662879 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:43.691944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 517/1038] 10.222.0.78 () {56 vars in 1133 bytes} [Wed May 25 14:42:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:43.701265 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a326450-d716-4679-a3e7-ef4da9284545 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:43.712790 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a749f1-41ca-489e-8170-3057fceaca9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:43.713956 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a749f1-41ca-489e-8170-3057fceaca9f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:43.714548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a749f1-41ca-489e-8170-3057fceaca9f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:43.715067 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a749f1-41ca-489e-8170-3057fceaca9f None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:43.716312 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a749f1-41ca-489e-8170-3057fceaca9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:43.718064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a749f1-41ca-489e-8170-3057fceaca9f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:43.738060 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 522/1039] 10.222.0.78 () {56 vars in 1087 bytes} [Wed May 25 14:42:43 2022] GET /identity/v3/services => generated 1957 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:42:43.753908 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:43.852114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a326450-d716-4679-a3e7-ef4da9284545 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:43.920114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c322b5-dda7-4b2c-b0d6-257015c22067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:43.920785 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c322b5-dda7-4b2c-b0d6-257015c22067 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:43.921062 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c322b5-dda7-4b2c-b0d6-257015c22067 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:43.921331 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c322b5-dda7-4b2c-b0d6-257015c22067 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:44.016926 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c322b5-dda7-4b2c-b0d6-257015c22067 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:44.019575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c322b5-dda7-4b2c-b0d6-257015c22067 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:44.084972 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:44.175478 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 518/1040] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:42:43 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 14:42:44.177432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a749f1-41ca-489e-8170-3057fceaca9f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:44.376603 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a749f1-41ca-489e-8170-3057fceaca9f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:44.443833 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923aee5-b5a6-4f77-9e60-999169ca40b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:44.445116 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923aee5-b5a6-4f77-9e60-999169ca40b8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:44.445603 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923aee5-b5a6-4f77-9e60-999169ca40b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:44.446200 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923aee5-b5a6-4f77-9e60-999169ca40b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:44.467128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d923aee5-b5a6-4f77-9e60-999169ca40b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:44.469882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d923aee5-b5a6-4f77-9e60-999169ca40b8 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:44.496401 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf7417-83c5-4254-ad92-c7a5ee7e315e None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:44.496957 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf7417-83c5-4254-ad92-c7a5ee7e315e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:44.496957 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf7417-83c5-4254-ad92-c7a5ee7e315e None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:44.502055 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 519/1041] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:44.503964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868ba7d-b953-4357-9f3c-159d2d65aa6d None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:44.504167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-d923aee5-b5a6-4f77-9e60-999169ca40b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:44.504167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:44.504167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:44.504167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:44.504167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:44.504167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:44.504167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:44.504167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:44.504167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:44.504167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:44.504167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:44.504167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:44.504167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:44.504167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:44.504167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:44.505347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:44.505347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:44.505347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:44.505347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:44.505347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:44.505347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:44.505347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:44.505347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:44.505347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:44.505347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:44.505347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:44.505347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:44.505347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:44.505347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:44.505347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:44.505347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:44.505347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:44.506625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:44.506625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:44.506625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:44.506625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:44.506941 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868ba7d-b953-4357-9f3c-159d2d65aa6d None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:44.506941 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868ba7d-b953-4357-9f3c-159d2d65aa6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:44.510200 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 523/1042] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:44.515308 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923aee5-b5a6-4f77-9e60-999169ca40b8 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:44.524536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-8868ba7d-b953-4357-9f3c-159d2d65aa6d None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:44.526973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-8868ba7d-b953-4357-9f3c-159d2d65aa6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:44.543291 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:44.643107 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923aee5-b5a6-4f77-9e60-999169ca40b8 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:44.650113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 520/1043] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:42:44.674478 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868ba7d-b953-4357-9f3c-159d2d65aa6d None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:44.696081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080c972-1f3d-40d9-9782-6afec39bba4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:44.696681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080c972-1f3d-40d9-9782-6afec39bba4f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:44.696970 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080c972-1f3d-40d9-9782-6afec39bba4f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:44.697251 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080c972-1f3d-40d9-9782-6afec39bba4f None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:44.698032 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080c972-1f3d-40d9-9782-6afec39bba4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:44.698372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080c972-1f3d-40d9-9782-6afec39bba4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:44.699798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080c972-1f3d-40d9-9782-6afec39bba4f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:44.713192 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 524/1044] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:44.719285 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080c972-1f3d-40d9-9782-6afec39bba4f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:44.750203 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:44.803261 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868ba7d-b953-4357-9f3c-159d2d65aa6d None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:44.845372 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8a479c-73f1-4af7-ae66-26f5a90b6267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:44.846169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a479c-73f1-4af7-ae66-26f5a90b6267 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:44.846376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a479c-73f1-4af7-ae66-26f5a90b6267 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:44.846648 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a479c-73f1-4af7-ae66-26f5a90b6267 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:44.859506 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8a479c-73f1-4af7-ae66-26f5a90b6267 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:42:44.864078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:44.867695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-ac8a479c-73f1-4af7-ae66-26f5a90b6267 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:42:44.867695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:44.867695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:44.867695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:44.867695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:44.867695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:44.867695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:44.867695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:44.867695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:44.867695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:44.867695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:44.867695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:44.867695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:42:44.867695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:42:44.867695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:42:44.867695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:42:44.874383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:42:44.874383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:42:44.874383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:42:44.874383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:42:44.874383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:44.874383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:44.874383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:44.874383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:44.874383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:44.874383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:44.874383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:44.874383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:44.874383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:44.874383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:44.874383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:44.874383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:44.875408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:44.875408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:44.875408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:44.875408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:44.875408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:42:44.875408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:42:44.875408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:42:44.875408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:42:44.875408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:42:44.875408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:42:44.875408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:42:44.875408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:44.875408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 521/1045] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:42:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:44.878091 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8a479c-73f1-4af7-ae66-26f5a90b6267 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:44.893280 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080c972-1f3d-40d9-9782-6afec39bba4f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:44.934705 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83a10d-cbc9-4be5-9830-971f0177cd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:44.935596 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83a10d-cbc9-4be5-9830-971f0177cd5b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:44.935888 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83a10d-cbc9-4be5-9830-971f0177cd5b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:44.936163 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83a10d-cbc9-4be5-9830-971f0177cd5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:44.944597 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83a10d-cbc9-4be5-9830-971f0177cd5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:44.946191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83a10d-cbc9-4be5-9830-971f0177cd5b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:44.962787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-0d83a10d-cbc9-4be5-9830-971f0177cd5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:44.962787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:44.962787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:44.962787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:44.962787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:44.962787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:44.962787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:44.962787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:44.962787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:44.962787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:44.962787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:44.962787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:44.962787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:42:44.962787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:42:44.962787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:42:44.964750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:42:44.964750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:44.964750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:44.964750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:44.964750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:44.964750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:44.964750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:44.964750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:44.964750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:44.964750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:44.964750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:44.964750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:44.964750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:44.964750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:44.964750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:44.964750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:44.964750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:44.965747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:42:44.965747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:42:44.965747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:42:44.965747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:42:44.965747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:42:44.965747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:42:44.965747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:44.965747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:44.965747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 525/1046] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:42:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:44.970666 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83a10d-cbc9-4be5-9830-971f0177cd5b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:45.034869 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8a479c-73f1-4af7-ae66-26f5a90b6267 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:45.069655 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:45.083802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb336fc8-2db1-4f07-aeb5-3475dcb57cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:45.084490 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb336fc8-2db1-4f07-aeb5-3475dcb57cb4 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:45.084768 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb336fc8-2db1-4f07-aeb5-3475dcb57cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:45.085042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb336fc8-2db1-4f07-aeb5-3475dcb57cb4 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:45.085914 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb336fc8-2db1-4f07-aeb5-3475dcb57cb4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:45.086160 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb336fc8-2db1-4f07-aeb5-3475dcb57cb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:45.087578 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb336fc8-2db1-4f07-aeb5-3475dcb57cb4 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:45.098163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 522/1047] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:42:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:45.106599 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb336fc8-2db1-4f07-aeb5-3475dcb57cb4 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:45.117357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83a10d-cbc9-4be5-9830-971f0177cd5b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:45.169942 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ac9a4-d1c5-4265-ad6e-5fa0d5d730f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:45.170677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ac9a4-d1c5-4265-ad6e-5fa0d5d730f5 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:45.170963 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ac9a4-d1c5-4265-ad6e-5fa0d5d730f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:45.171239 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ac9a4-d1c5-4265-ad6e-5fa0d5d730f5 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:45.172040 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ac9a4-d1c5-4265-ad6e-5fa0d5d730f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:45.172358 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ac9a4-d1c5-4265-ad6e-5fa0d5d730f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:45.173869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ac9a4-d1c5-4265-ad6e-5fa0d5d730f5 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:45.197994 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 526/1048] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:42:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:45.201824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ac9a4-d1c5-4265-ad6e-5fa0d5d730f5 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:45.205944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:45.260306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb336fc8-2db1-4f07-aeb5-3475dcb57cb4 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:45.304975 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a34277-6bea-44c6-9a36-f1fac1cd135a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:45.306760 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a34277-6bea-44c6-9a36-f1fac1cd135a None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:45.307036 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a34277-6bea-44c6-9a36-f1fac1cd135a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:45.307315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a34277-6bea-44c6-9a36-f1fac1cd135a None admin] PATH_INFO: `/v3/users/1712d7d84c8347c78e7385fc1bea0dcb` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:45.339742 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a34277-6bea-44c6-9a36-f1fac1cd135a None admin] RBAC: Authorizing `identity:get_user(user_id=1712d7d84c8347c78e7385fc1bea0dcb)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:45.340961 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ac9a4-d1c5-4265-ad6e-5fa0d5d730f5 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:45.342582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a34277-6bea-44c6-9a36-f1fac1cd135a None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:45.346240 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 523/1049] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:42:45 2022] GET /identity/v3/users/1712d7d84c8347c78e7385fc1bea0dcb => generated 285 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:45.351591 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a34277-6bea-44c6-9a36-f1fac1cd135a None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:45.391430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709ea7c-f5bc-487e-8f90-a584da45488a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:45.392113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709ea7c-f5bc-487e-8f90-a584da45488a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:45.392395 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709ea7c-f5bc-487e-8f90-a584da45488a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:45.392670 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709ea7c-f5bc-487e-8f90-a584da45488a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:45.393543 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709ea7c-f5bc-487e-8f90-a584da45488a None admin] RBAC: Adding query filter params (role.id=074ec0dc23ca41fbb4833f8857859512, scope.project.id=908a13d2b49544fbaf760b07e213bb7d, user.id=e34e1d7ffbac49b0bde45829af0c6925) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:45.393858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709ea7c-f5bc-487e-8f90-a584da45488a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:45.395439 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709ea7c-f5bc-487e-8f90-a584da45488a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:45.410050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 527/1050] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:42:45 2022] GET /identity/v3/role_assignments?user.id=e34e1d7ffbac49b0bde45829af0c6925&scope.project.id=908a13d2b49544fbaf760b07e213bb7d&role.id=074ec0dc23ca41fbb4833f8857859512 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:45.417741 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:45.502658 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a34277-6bea-44c6-9a36-f1fac1cd135a None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:45.567122 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22982944-092a-49ca-9d38-cf001add2969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:45.568119 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22982944-092a-49ca-9d38-cf001add2969 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:45.568401 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22982944-092a-49ca-9d38-cf001add2969 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:45.568763 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22982944-092a-49ca-9d38-cf001add2969 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:45.677000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22982944-092a-49ca-9d38-cf001add2969 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:45.678744 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22982944-092a-49ca-9d38-cf001add2969 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:45.740363 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:45.843425 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 524/1051] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:42:45 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 14:42:45.850374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709ea7c-f5bc-487e-8f90-a584da45488a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:45.959389 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:46.032872 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709ea7c-f5bc-487e-8f90-a584da45488a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:46.103485 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076692e5-9912-4fee-83e4-1bd496191d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:46.104982 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076692e5-9912-4fee-83e4-1bd496191d51 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:46.105496 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076692e5-9912-4fee-83e4-1bd496191d51 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:46.106020 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076692e5-9912-4fee-83e4-1bd496191d51 None admin] PATH_INFO: `/v3/projects/6418c93485d7455688abb66e3431cac3` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:46.108173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076692e5-9912-4fee-83e4-1bd496191d51 None admin] RBAC: Authorizing `identity:get_project(project_id=6418c93485d7455688abb66e3431cac3)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:46.111046 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076692e5-9912-4fee-83e4-1bd496191d51 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:46.114484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 528/1052] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:42:45 2022] GET /identity/v3/projects/6418c93485d7455688abb66e3431cac3 => generated 311 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:46.123219 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22982944-092a-49ca-9d38-cf001add2969 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:46.311486 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22982944-092a-49ca-9d38-cf001add2969 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:46.376123 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88cc57-4faf-4d25-aee6-824126f34649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:46.377144 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88cc57-4faf-4d25-aee6-824126f34649 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:46.377606 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88cc57-4faf-4d25-aee6-824126f34649 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:46.378101 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88cc57-4faf-4d25-aee6-824126f34649 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:46.379739 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88cc57-4faf-4d25-aee6-824126f34649 None admin] RBAC: Adding query filter params (role.id=e566b37cd92f467198b5dc40f183effc, scope.project.id=6418c93485d7455688abb66e3431cac3, user.id=1712d7d84c8347c78e7385fc1bea0dcb) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:46.380283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88cc57-4faf-4d25-aee6-824126f34649 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:46.383044 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88cc57-4faf-4d25-aee6-824126f34649 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:46.402422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 525/1053] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:42:46 2022] GET /identity/v3/role_assignments?user.id=1712d7d84c8347c78e7385fc1bea0dcb&scope.project.id=6418c93485d7455688abb66e3431cac3&role.id=e566b37cd92f467198b5dc40f183effc => generated 259 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:46.483514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5411306-7b71-4aa7-8262-6ae806a80461 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:46.483865 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5411306-7b71-4aa7-8262-6ae806a80461 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:46.484157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5411306-7b71-4aa7-8262-6ae806a80461 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:46.485950 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 529/1054] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:46.492184 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7ff33-57c3-41ef-8b98-58f09aca76a0 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:46.492463 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7ff33-57c3-41ef-8b98-58f09aca76a0 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:46.492984 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7ff33-57c3-41ef-8b98-58f09aca76a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:46.512561 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-35e7ff33-57c3-41ef-8b98-58f09aca76a0 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:46.514451 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-35e7ff33-57c3-41ef-8b98-58f09aca76a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:46.516942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:46.602425 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 526/1055] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:46.626490 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5411306-7b71-4aa7-8262-6ae806a80461 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:46.751557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5411306-7b71-4aa7-8262-6ae806a80461 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:46.792437 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4059e-f086-41fb-b35b-588c6121c330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:46.793278 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4059e-f086-41fb-b35b-588c6121c330 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:46.793591 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4059e-f086-41fb-b35b-588c6121c330 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:46.793859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4059e-f086-41fb-b35b-588c6121c330 None admin] PATH_INFO: `/v3/services/image` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:46.794569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4059e-f086-41fb-b35b-588c6121c330 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:46.796106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4059e-f086-41fb-b35b-588c6121c330 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:46.798931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:46.812385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-6ab4059e-f086-41fb-b35b-588c6121c330 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:42:46.812385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:46.812385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:42:46.812385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:42:46.812385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:42:46.812385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:42:46.812385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:42:46.812385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:46.812385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:42:46.812385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:46.812385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:42:46.812385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:46.812385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:46.812385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:46.812385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:46.812385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:46.812385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:46.813605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:46.813605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:46.813605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:46.813605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:46.813605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:46.813605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:46.813605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:42:46.813605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:42:46.813605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:42:46.813605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:42:46.813605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:46.813605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:46.813605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:46.813605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:46.813605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:46.813605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:46.814658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:46.814658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:46.814658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:46.814658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:46.814658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:46.814658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:46.814658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:46.814658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:46.814658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:46.814658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:46.814658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:42:46.814658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:42:46.814658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:42:46.814658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:46.814658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 530/1056] 10.222.0.78 () {56 vars in 1105 bytes} [Wed May 25 14:42:46 2022] GET /identity/v3/services/image => generated 86 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:46.822722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7ff33-57c3-41ef-8b98-58f09aca76a0 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:46.975634 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7ff33-57c3-41ef-8b98-58f09aca76a0 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:47.028758 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230acdb0-730c-4242-9bdd-01dc2d2364f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:47.029440 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230acdb0-730c-4242-9bdd-01dc2d2364f1 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:47.029713 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230acdb0-730c-4242-9bdd-01dc2d2364f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:47.029990 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230acdb0-730c-4242-9bdd-01dc2d2364f1 None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:47.030745 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230acdb0-730c-4242-9bdd-01dc2d2364f1 None admin] RBAC: Adding query filter params (name=image) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:47.031068 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230acdb0-730c-4242-9bdd-01dc2d2364f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:47.032455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230acdb0-730c-4242-9bdd-01dc2d2364f1 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:47.044631 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 527/1057] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:47.045886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4059e-f086-41fb-b35b-588c6121c330 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:47.183185 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4059e-f086-41fb-b35b-588c6121c330 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:47.232417 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a8bdc-ef52-4989-ab06-e5c2c0c540c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:47.233811 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a8bdc-ef52-4989-ab06-e5c2c0c540c4 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:47.234114 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a8bdc-ef52-4989-ab06-e5c2c0c540c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:47.234394 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a8bdc-ef52-4989-ab06-e5c2c0c540c4 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:47.235127 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a8bdc-ef52-4989-ab06-e5c2c0c540c4 None admin] RBAC: Adding query filter params (type=image) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:47.235430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a8bdc-ef52-4989-ab06-e5c2c0c540c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:47.236860 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a8bdc-ef52-4989-ab06-e5c2c0c540c4 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:47.243140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 531/1058] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:47.250157 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230acdb0-730c-4242-9bdd-01dc2d2364f1 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:47.254837 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:47.392996 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230acdb0-730c-4242-9bdd-01dc2d2364f1 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:47.439109 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b033816-2426-4cd7-aeaa-095b63c05a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:47.439827 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b033816-2426-4cd7-aeaa-095b63c05a23 None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:47.440130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b033816-2426-4cd7-aeaa-095b63c05a23 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:47.440463 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b033816-2426-4cd7-aeaa-095b63c05a23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:47.441240 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b033816-2426-4cd7-aeaa-095b63c05a23 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:47.442848 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b033816-2426-4cd7-aeaa-095b63c05a23 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:47.466068 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 528/1059] 10.222.0.78 () {60 vars in 1144 bytes} [Wed May 25 14:42:47 2022] POST /identity/v3/endpoints => generated 327 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:42:47.757937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c9b60-86a4-4027-90cf-875035291d42 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:47.758277 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c9b60-86a4-4027-90cf-875035291d42 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:47.758686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c9b60-86a4-4027-90cf-875035291d42 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:47.760132 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 532/1060] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:42:47.765887 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638d60c-736a-4261-ab87-0eddbb6a3504 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:47.766782 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638d60c-736a-4261-ab87-0eddbb6a3504 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:47.766782 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638d60c-736a-4261-ab87-0eddbb6a3504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:47.785292 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-3638d60c-736a-4261-ab87-0eddbb6a3504 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:47.786526 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-3638d60c-736a-4261-ab87-0eddbb6a3504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:47.874815 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:47.916422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 529/1061] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:42:47.938748 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c9b60-86a4-4027-90cf-875035291d42 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:48.013368 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:48.064434 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c9b60-86a4-4027-90cf-875035291d42 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:48.117642 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4257e0d-2ab0-46b6-8966-94c9653f9beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:48.118496 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4257e0d-2ab0-46b6-8966-94c9653f9beb None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:48.118779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4257e0d-2ab0-46b6-8966-94c9653f9beb None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:48.119051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4257e0d-2ab0-46b6-8966-94c9653f9beb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:48.139468 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4257e0d-2ab0-46b6-8966-94c9653f9beb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:42:48.147793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-f4257e0d-2ab0-46b6-8966-94c9653f9beb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:42:48.147793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:48.147793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:48.147793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:48.147793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:48.147793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:48.147793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:48.147793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:48.147793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:48.147793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:48.147793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:48.147793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:48.147793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:42:48.147793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:42:48.147793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:42:48.147793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:42:48.149050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:42:48.149050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:42:48.149050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:42:48.149050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:42:48.149050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:48.149050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:48.149050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:48.149050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:48.149050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:48.149050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:48.149050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:48.149050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:48.149050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:48.149050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:48.149050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:48.149050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:48.150075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:48.150075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:48.150075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:48.150075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:48.150075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:42:48.150075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:42:48.150075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:42:48.150075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:42:48.150075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:42:48.150075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:42:48.150075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:42:48.150075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:48.154479 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 533/1062] 10.222.0.78 () {56 vars in 1120 bytes} [Wed May 25 14:42:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:48.155854 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638d60c-736a-4261-ab87-0eddbb6a3504 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:48.334639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638d60c-736a-4261-ab87-0eddbb6a3504 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:48.379954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29eda3-0010-4676-a7c0-062f9cf34cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:48.380631 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29eda3-0010-4676-a7c0-062f9cf34cd0 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:48.381060 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29eda3-0010-4676-a7c0-062f9cf34cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:48.381475 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29eda3-0010-4676-a7c0-062f9cf34cd0 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:48.382355 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29eda3-0010-4676-a7c0-062f9cf34cd0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:48.382705 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29eda3-0010-4676-a7c0-062f9cf34cd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:48.384107 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29eda3-0010-4676-a7c0-062f9cf34cd0 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:48.395342 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 530/1063] 10.222.0.78 () {56 vars in 1115 bytes} [Wed May 25 14:42:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:48.401376 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4257e0d-2ab0-46b6-8966-94c9653f9beb None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:48.544612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4257e0d-2ab0-46b6-8966-94c9653f9beb None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:48.573922 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:48.588849 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b40744-ae45-41d0-ab3e-9decfaf3a9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:48.590171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b40744-ae45-41d0-ab3e-9decfaf3a9a0 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:48.590534 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b40744-ae45-41d0-ab3e-9decfaf3a9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:48.590812 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b40744-ae45-41d0-ab3e-9decfaf3a9a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:48.604494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b40744-ae45-41d0-ab3e-9decfaf3a9a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:48.606576 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b40744-ae45-41d0-ab3e-9decfaf3a9a0 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:48.640113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-52b40744-ae45-41d0-ab3e-9decfaf3a9a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:48.640113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:48.640113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:48.640113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:48.640113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:48.640113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:48.640113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:48.640113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:48.640113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:48.640113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:48.640113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:48.640113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:48.640113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:48.640113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:48.640113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:48.641094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:48.641094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:48.641094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:48.641094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:48.641094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:48.641094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:48.641094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:48.641094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:48.641094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:48.641094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:48.641094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:48.641094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:48.641094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:48.641094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:48.641094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:48.641094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:48.641094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:48.641992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:48.641992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:48.641992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:48.641992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:48.649439 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 534/1064] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:48.650729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29eda3-0010-4676-a7c0-062f9cf34cd0 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:48.711939 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:48.743493 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84933689-6315-4e39-afda-49c2de7b4734 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:48.743839 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84933689-6315-4e39-afda-49c2de7b4734 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:48.744068 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84933689-6315-4e39-afda-49c2de7b4734 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:48.745635 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 535/1065] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:48.751445 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6423e-dee9-4d38-9e9f-672967f0caf6 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:48.751788 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6423e-dee9-4d38-9e9f-672967f0caf6 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:48.751986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6423e-dee9-4d38-9e9f-672967f0caf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:48.771354 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-38a6423e-dee9-4d38-9e9f-672967f0caf6 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:48.772655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-38a6423e-dee9-4d38-9e9f-672967f0caf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:48.788991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29eda3-0010-4676-a7c0-062f9cf34cd0 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:48.840979 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbfca0d-d37a-478f-80ad-ea2e7708daa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:48.842024 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfca0d-d37a-478f-80ad-ea2e7708daa2 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:48.842320 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfca0d-d37a-478f-80ad-ea2e7708daa2 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:48.842625 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfca0d-d37a-478f-80ad-ea2e7708daa2 None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:48.843541 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbfca0d-d37a-478f-80ad-ea2e7708daa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:48.844042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbfca0d-d37a-478f-80ad-ea2e7708daa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:48.846200 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbfca0d-d37a-478f-80ad-ea2e7708daa2 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:48.855719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 536/1066] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:42:48.861212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 531/1067] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:48.866747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6423e-dee9-4d38-9e9f-672967f0caf6 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:48.897130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbfca0d-d37a-478f-80ad-ea2e7708daa2 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:49.015178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6423e-dee9-4d38-9e9f-672967f0caf6 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:49.019453 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:49.043174 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbfca0d-d37a-478f-80ad-ea2e7708daa2 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:49.052444 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:49.066133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0631e99-8af8-43be-bab7-a6599d875ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:49.066950 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0631e99-8af8-43be-bab7-a6599d875ef5 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:49.067228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0631e99-8af8-43be-bab7-a6599d875ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:49.067507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0631e99-8af8-43be-bab7-a6599d875ef5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:49.076257 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0631e99-8af8-43be-bab7-a6599d875ef5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:49.077730 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0631e99-8af8-43be-bab7-a6599d875ef5 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:49.091477 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17329346-5133-4c04-a16d-2a3afff6ea03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:49.092251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17329346-5133-4c04-a16d-2a3afff6ea03 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:49.092534 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17329346-5133-4c04-a16d-2a3afff6ea03 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:49.092806 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17329346-5133-4c04-a16d-2a3afff6ea03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:49.094298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-a0631e99-8af8-43be-bab7-a6599d875ef5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:42:49.094298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:49.094298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:49.094298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:49.094298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:49.094298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:49.094298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:49.094298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:49.094298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:49.094298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:49.094298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:49.094298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:49.094298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:42:49.094298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:42:49.094298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:42:49.094298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:42:49.095421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:49.095421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:49.095421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:42:49.095421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:49.095421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:42:49.095421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:49.095421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:49.095421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:49.095421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:49.095421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:49.095421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:49.095421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:49.095421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:49.095421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:49.095421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:49.095421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:49.095421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:49.095421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:49.096649 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:49.096649 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:49.096649 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:42:49.096649 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:42:49.096649 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:42:49.096649 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:42:49.096649 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:42:49.096649 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:42:49.096649 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:42:49.096649 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:49.096649 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 537/1068] 10.222.0.78 () {56 vars in 1093 bytes} [Wed May 25 14:42:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:49.102307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0631e99-8af8-43be-bab7-a6599d875ef5 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:49.109360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17329346-5133-4c04-a16d-2a3afff6ea03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:42:49.117208 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-17329346-5133-4c04-a16d-2a3afff6ea03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:42:49.117208 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:49.117208 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:49.117208 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:49.117208 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:49.117208 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:49.117208 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:49.117208 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:49.117208 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:49.117208 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:49.117208 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:49.117208 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:49.117208 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:42:49.117208 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:42:49.117208 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:42:49.117208 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:42:49.118362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:42:49.118362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:42:49.118362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:42:49.118362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:42:49.118362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:49.118362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:49.118362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:49.118362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:49.118362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:49.118362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:49.118362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:49.118362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:42:49.118362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:49.118362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:49.118362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:49.118362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:49.119413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:49.119413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:49.119413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:49.119413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:49.119413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:42:49.119413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:42:49.119413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:42:49.119413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:42:49.119413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:42:49.119413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:42:49.119413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:42:49.119413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:42:49.119413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 532/1069] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:42:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:49.125638 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17329346-5133-4c04-a16d-2a3afff6ea03 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:49.256884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17329346-5133-4c04-a16d-2a3afff6ea03 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:49.272704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0631e99-8af8-43be-bab7-a6599d875ef5 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:49.304498 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323dbc65-5979-4497-aabb-34ad3e5e72b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:49.305803 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323dbc65-5979-4497-aabb-34ad3e5e72b0 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:49.306137 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323dbc65-5979-4497-aabb-34ad3e5e72b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:49.306411 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323dbc65-5979-4497-aabb-34ad3e5e72b0 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:49.307236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323dbc65-5979-4497-aabb-34ad3e5e72b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:49.308154 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323dbc65-5979-4497-aabb-34ad3e5e72b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:49.309545 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323dbc65-5979-4497-aabb-34ad3e5e72b0 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:49.318611 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf94cd-07fe-4d13-ba55-271662b2cea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:49.319383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 533/1070] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:42:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:49.319538 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf94cd-07fe-4d13-ba55-271662b2cea4 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:49.319779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf94cd-07fe-4d13-ba55-271662b2cea4 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:49.320230 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf94cd-07fe-4d13-ba55-271662b2cea4 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:49.321065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf94cd-07fe-4d13-ba55-271662b2cea4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:49.321384 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf94cd-07fe-4d13-ba55-271662b2cea4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:49.322943 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf94cd-07fe-4d13-ba55-271662b2cea4 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:49.325115 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323dbc65-5979-4497-aabb-34ad3e5e72b0 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:49.353934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 538/1071] 10.222.0.78 () {56 vars in 1133 bytes} [Wed May 25 14:42:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:49.359612 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:49.360385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf94cd-07fe-4d13-ba55-271662b2cea4 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:49.398240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:49.499622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323dbc65-5979-4497-aabb-34ad3e5e72b0 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:49.500168 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf94cd-07fe-4d13-ba55-271662b2cea4 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:49.545013 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cdbb17-331a-4f91-b3c1-8a3ba98bd772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:49.545962 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cdbb17-331a-4f91-b3c1-8a3ba98bd772 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:49.546272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cdbb17-331a-4f91-b3c1-8a3ba98bd772 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:49.546554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cdbb17-331a-4f91-b3c1-8a3ba98bd772 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:49.561827 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cdbb17-331a-4f91-b3c1-8a3ba98bd772 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:49.563388 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cdbb17-331a-4f91-b3c1-8a3ba98bd772 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:49.572785 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8a5ce-c6c0-4b63-90c5-d9855bc5de61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:49.576164 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8a5ce-c6c0-4b63-90c5-d9855bc5de61 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:49.576678 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8a5ce-c6c0-4b63-90c5-d9855bc5de61 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:49.577186 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8a5ce-c6c0-4b63-90c5-d9855bc5de61 None admin] PATH_INFO: `/v3/users/1712d7d84c8347c78e7385fc1bea0dcb` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:49.579624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e8a5ce-c6c0-4b63-90c5-d9855bc5de61 None admin] RBAC: Authorizing `identity:get_user(user_id=1712d7d84c8347c78e7385fc1bea0dcb)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:49.582488 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e8a5ce-c6c0-4b63-90c5-d9855bc5de61 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:49.585744 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 534/1072] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:42:49 2022] GET /identity/v3/users/1712d7d84c8347c78e7385fc1bea0dcb => generated 285 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:49.585995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-d6cdbb17-331a-4f91-b3c1-8a3ba98bd772 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:49.585995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:49.585995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:49.585995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:49.585995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:49.585995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:49.585995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:49.585995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:49.585995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:49.585995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:49.585995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:49.585995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:49.585995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:49.585995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:49.585995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:49.587739 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:49.587739 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:49.587739 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:49.587739 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:49.587739 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:49.587739 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:49.587739 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:49.587739 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:49.587739 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:49.587739 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:49.587739 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:49.587739 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:49.587739 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:49.587739 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:49.587739 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:49.587739 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:49.587739 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:49.590172 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:49.590172 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:49.590172 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:49.590172 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:49.592291 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 539/1073] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:49.593224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cdbb17-331a-4f91-b3c1-8a3ba98bd772 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:49.593418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8a5ce-c6c0-4b63-90c5-d9855bc5de61 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:49.710830 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:49.752815 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:49.761720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cdbb17-331a-4f91-b3c1-8a3ba98bd772 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:49.768832 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8a5ce-c6c0-4b63-90c5-d9855bc5de61 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:49.816890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a21c0-c8e3-48d0-91a2-0bf3a6971148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:49.817934 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a21c0-c8e3-48d0-91a2-0bf3a6971148 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:49.818117 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a21c0-c8e3-48d0-91a2-0bf3a6971148 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:49.818471 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a21c0-c8e3-48d0-91a2-0bf3a6971148 None admin] PATH_INFO: `/v3/projects/6418c93485d7455688abb66e3431cac3` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:49.819926 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a21c0-c8e3-48d0-91a2-0bf3a6971148 None admin] RBAC: Authorizing `identity:get_project(project_id=6418c93485d7455688abb66e3431cac3)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:49.822226 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a21c0-c8e3-48d0-91a2-0bf3a6971148 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:49.824966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 535/1074] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:42:49 2022] GET /identity/v3/projects/6418c93485d7455688abb66e3431cac3 => generated 311 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:49.829370 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7008b-0041-4d97-a091-37a80d821d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:49.830784 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a21c0-c8e3-48d0-91a2-0bf3a6971148 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:49.830957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7008b-0041-4d97-a091-37a80d821d60 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:49.831447 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7008b-0041-4d97-a091-37a80d821d60 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:49.831939 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7008b-0041-4d97-a091-37a80d821d60 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:49.833299 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7008b-0041-4d97-a091-37a80d821d60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:49.833846 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7008b-0041-4d97-a091-37a80d821d60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:49.836620 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7008b-0041-4d97-a091-37a80d821d60 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:49.862881 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 540/1075] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:49.865586 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7008b-0041-4d97-a091-37a80d821d60 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:49.990955 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a21c0-c8e3-48d0-91a2-0bf3a6971148 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:50.002833 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7008b-0041-4d97-a091-37a80d821d60 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:50.041812 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b91f1d-6111-4e2f-a723-613d4a2edbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:50.042655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b91f1d-6111-4e2f-a723-613d4a2edbe2 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:50.042942 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b91f1d-6111-4e2f-a723-613d4a2edbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:50.043226 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b91f1d-6111-4e2f-a723-613d4a2edbe2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:50.045544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:50.046582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb109206-1ccf-465a-953a-01e4d95e5b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:50.048182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb109206-1ccf-465a-953a-01e4d95e5b2a None admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:50.048695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb109206-1ccf-465a-953a-01e4d95e5b2a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:50.049195 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb109206-1ccf-465a-953a-01e4d95e5b2a None admin] PATH_INFO: `/v3/projects/6418c93485d7455688abb66e3431cac3/users/1712d7d84c8347c78e7385fc1bea0dcb/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:50.052128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b91f1d-6111-4e2f-a723-613d4a2edbe2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:50.052970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb109206-1ccf-465a-953a-01e4d95e5b2a None admin] RBAC: Authorizing `identity:create_grant(project_id=6418c93485d7455688abb66e3431cac3, user_id=1712d7d84c8347c78e7385fc1bea0dcb, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:50.053897 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b91f1d-6111-4e2f-a723-613d4a2edbe2 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:50.055796 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb109206-1ccf-465a-953a-01e4d95e5b2a None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:50.070353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-e4b91f1d-6111-4e2f-a723-613d4a2edbe2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:50.070353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:50.070353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:50.070353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:50.070353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:50.070353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:50.070353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:50.070353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:50.070353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:50.070353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:50.070353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:50.070353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:50.070353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:42:50.070353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:42:50.070353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:42:50.072873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:42:50.072873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:50.072873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:50.072873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:50.072873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:50.072873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:50.072873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:50.072873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:50.072873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:50.072873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:50.072873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:50.072873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:50.072873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:50.072873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:50.072873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:50.072873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:50.072873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:50.082690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:42:50.082690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:42:50.082690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:42:50.082690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:42:50.082690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:42:50.082690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:42:50.082690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:50.082690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:50.082690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 541/1076] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:42:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:50.082690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51e2ff-a9b1-4273-a623-86e39b35fcc2 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:50.082690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51e2ff-a9b1-4273-a623-86e39b35fcc2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:50.085005 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51e2ff-a9b1-4273-a623-86e39b35fcc2 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:50.085005 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 542/1077] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:50.085005 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51e2ff-a9b1-4273-a623-86e39b35fcc2 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:50.085687 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 536/1078] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:42:49 2022] PUT /identity/v3/projects/6418c93485d7455688abb66e3431cac3/users/1712d7d84c8347c78e7385fc1bea0dcb/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:42:50.085687 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376cd95-80cb-46a4-a270-fb4e1ebf73da None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:50.085687 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376cd95-80cb-46a4-a270-fb4e1ebf73da None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:50.085687 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376cd95-80cb-46a4-a270-fb4e1ebf73da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:50.110558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-3376cd95-80cb-46a4-a270-fb4e1ebf73da None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:50.111978 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-3376cd95-80cb-46a4-a270-fb4e1ebf73da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:50.141424 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:50.239863 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 537/1079] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:50.258777 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51e2ff-a9b1-4273-a623-86e39b35fcc2 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:50.266971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376cd95-80cb-46a4-a270-fb4e1ebf73da None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:50.320167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4c600-23f9-45b5-ade2-234c3d3a4dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:50.321234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4c600-23f9-45b5-ade2-234c3d3a4dcf None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:50.321711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4c600-23f9-45b5-ade2-234c3d3a4dcf None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:50.322260 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4c600-23f9-45b5-ade2-234c3d3a4dcf None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:50.324112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e4c600-23f9-45b5-ade2-234c3d3a4dcf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:50.324629 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e4c600-23f9-45b5-ade2-234c3d3a4dcf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:50.327289 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e4c600-23f9-45b5-ade2-234c3d3a4dcf None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:50.354372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 543/1080] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:42:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:50.357218 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4c600-23f9-45b5-ade2-234c3d3a4dcf None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:50.426722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376cd95-80cb-46a4-a270-fb4e1ebf73da None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:50.456903 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:50.491835 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90098578-aa50-46fd-82c5-e1f33be253cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:50.492492 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90098578-aa50-46fd-82c5-e1f33be253cf None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:50.492781 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90098578-aa50-46fd-82c5-e1f33be253cf None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:50.493031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90098578-aa50-46fd-82c5-e1f33be253cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:50.516633 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4c600-23f9-45b5-ade2-234c3d3a4dcf None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:50.533891 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:50.561601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a2bca-9652-4987-9a12-4bc5e2349967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:50.562546 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a2bca-9652-4987-9a12-4bc5e2349967 None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:50.562826 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a2bca-9652-4987-9a12-4bc5e2349967 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:50.563105 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a2bca-9652-4987-9a12-4bc5e2349967 None admin] PATH_INFO: `/v3/projects/908a13d2b49544fbaf760b07e213bb7d/users/e34e1d7ffbac49b0bde45829af0c6925/roles/074ec0dc23ca41fbb4833f8857859512` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:50.565463 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a2bca-9652-4987-9a12-4bc5e2349967 None admin] RBAC: Authorizing `identity:create_grant(project_id=908a13d2b49544fbaf760b07e213bb7d, user_id=e34e1d7ffbac49b0bde45829af0c6925, role_id=074ec0dc23ca41fbb4833f8857859512)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:50.566987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a2bca-9652-4987-9a12-4bc5e2349967 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:50.583196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 544/1081] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:42:50 2022] PUT /identity/v3/projects/908a13d2b49544fbaf760b07e213bb7d/users/e34e1d7ffbac49b0bde45829af0c6925/roles/074ec0dc23ca41fbb4833f8857859512 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:42:50.627561 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90098578-aa50-46fd-82c5-e1f33be253cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:50.631091 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90098578-aa50-46fd-82c5-e1f33be253cf None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:50.781965 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 538/1082] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:42:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 14:42:50.788882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a2bca-9652-4987-9a12-4bc5e2349967 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:50.934749 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a2bca-9652-4987-9a12-4bc5e2349967 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:50.952659 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:50.989539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2818671-d895-4b3c-981f-ef627e5cd40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:50.990879 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2818671-d895-4b3c-981f-ef627e5cd40e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:50.991392 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2818671-d895-4b3c-981f-ef627e5cd40e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:50.991887 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2818671-d895-4b3c-981f-ef627e5cd40e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:51.014131 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2818671-d895-4b3c-981f-ef627e5cd40e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:51.017007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2818671-d895-4b3c-981f-ef627e5cd40e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:51.039884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-d2818671-d895-4b3c-981f-ef627e5cd40e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:51.039884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:51.039884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:51.039884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:51.039884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:51.039884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:51.039884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:51.039884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:51.039884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:51.039884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:51.039884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:51.039884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:51.039884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:51.039884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:51.039884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:51.041253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:51.041253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:51.041253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:51.041253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:51.041253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:51.041253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:51.041253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:51.041253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:51.041253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:51.041253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:51.041253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:51.041253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:51.041253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:51.041253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:51.041253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:51.041253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:51.041253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:51.043035 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:51.043035 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:51.043035 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:51.043035 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:51.046342 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 545/1083] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:51.047458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90098578-aa50-46fd-82c5-e1f33be253cf None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:51.133962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:51.231267 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90098578-aa50-46fd-82c5-e1f33be253cf None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:51.299014 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ede28-1b58-4aad-ae6b-4e2f3f13d07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:51.300055 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ede28-1b58-4aad-ae6b-4e2f3f13d07a None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:51.300564 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ede28-1b58-4aad-ae6b-4e2f3f13d07a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:51.301055 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ede28-1b58-4aad-ae6b-4e2f3f13d07a None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:51.302516 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ede28-1b58-4aad-ae6b-4e2f3f13d07a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:51.303067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ede28-1b58-4aad-ae6b-4e2f3f13d07a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:51.305744 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ede28-1b58-4aad-ae6b-4e2f3f13d07a None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:51.330127 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 539/1084] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:52.609498 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d9faf-d514-4c50-bd10-1101daa1b1b3 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:52.610406 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d9faf-d514-4c50-bd10-1101daa1b1b3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:52.610406 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d9faf-d514-4c50-bd10-1101daa1b1b3 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:52.611479 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 546/1085] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:42:52.621776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ceef00-2c2f-4260-8701-ccf7f8223ad3 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:52.622372 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ceef00-2c2f-4260-8701-ccf7f8223ad3 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:52.622764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ceef00-2c2f-4260-8701-ccf7f8223ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:52.645860 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-f7ceef00-2c2f-4260-8701-ccf7f8223ad3 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:52.647107 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f7ceef00-2c2f-4260-8701-ccf7f8223ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:52.753428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 540/1086] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:52.793822 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d9faf-d514-4c50-bd10-1101daa1b1b3 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:52.992190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d9faf-d514-4c50-bd10-1101daa1b1b3 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:53.045217 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c27ebf-0231-42ca-9b64-c628b9fa1a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:53.046037 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c27ebf-0231-42ca-9b64-c628b9fa1a89 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:53.046331 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c27ebf-0231-42ca-9b64-c628b9fa1a89 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:53.046607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c27ebf-0231-42ca-9b64-c628b9fa1a89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:53.058794 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c27ebf-0231-42ca-9b64-c628b9fa1a89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:42:53.066569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-c1c27ebf-0231-42ca-9b64-c628b9fa1a89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:42:53.066569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:53.066569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:53.066569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:53.066569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:53.066569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:53.066569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:53.066569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:53.066569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:53.066569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:53.066569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:53.066569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:53.066569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:42:53.066569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:42:53.066569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:42:53.066569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:42:53.068028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:42:53.068028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:42:53.068028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:42:53.068028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:42:53.068028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:53.068028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:53.068028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:53.068028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:53.068028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:53.068028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:53.068028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:53.068028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:53.068028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:53.068028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:53.068028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:53.068028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:53.069459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:53.069459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:53.069459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:53.069459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:53.069459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:42:53.069459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:42:53.069459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:42:53.069459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:42:53.069459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:42:53.069459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:42:53.069459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:42:53.069459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:53.069459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 547/1087] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:42:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:53.075447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ceef00-2c2f-4260-8701-ccf7f8223ad3 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:53.076402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe543d91-edaa-4621-b80c-40983162d1cc None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:53.076698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe543d91-edaa-4621-b80c-40983162d1cc None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:53.076977 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe543d91-edaa-4621-b80c-40983162d1cc None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:53.078636 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 548/1088] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:42:53.084324 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9be80a-b9d8-4558-b285-6c17c79ac86d None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:53.084609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9be80a-b9d8-4558-b285-6c17c79ac86d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:53.084893 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9be80a-b9d8-4558-b285-6c17c79ac86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:53.103446 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-9f9be80a-b9d8-4558-b285-6c17c79ac86d None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:53.104688 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9f9be80a-b9d8-4558-b285-6c17c79ac86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:53.129010 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:53.150611 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:53.187494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 549/1089] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:42:53.219975 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9be80a-b9d8-4558-b285-6c17c79ac86d None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:53.221246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ceef00-2c2f-4260-8701-ccf7f8223ad3 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:53.274510 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49ec66-9831-4fdc-859e-b168d8831ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:53.275195 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49ec66-9831-4fdc-859e-b168d8831ad0 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:53.275477 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49ec66-9831-4fdc-859e-b168d8831ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:53.275751 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49ec66-9831-4fdc-859e-b168d8831ad0 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:53.278407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49ec66-9831-4fdc-859e-b168d8831ad0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:53.278739 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49ec66-9831-4fdc-859e-b168d8831ad0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:53.280674 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49ec66-9831-4fdc-859e-b168d8831ad0 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:53.292714 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 541/1090] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:42:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:53.300089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49ec66-9831-4fdc-859e-b168d8831ad0 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:53.365702 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9be80a-b9d8-4558-b285-6c17c79ac86d None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:53.408134 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4beda-fd86-4218-9b94-e7b6fefbaad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:53.409279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4beda-fd86-4218-9b94-e7b6fefbaad3 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:53.409573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4beda-fd86-4218-9b94-e7b6fefbaad3 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:53.409846 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4beda-fd86-4218-9b94-e7b6fefbaad3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:53.422581 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4beda-fd86-4218-9b94-e7b6fefbaad3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:42:53.430981 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-a4f4beda-fd86-4218-9b94-e7b6fefbaad3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:42:53.430981 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:53.430981 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:53.430981 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:53.430981 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:53.430981 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:53.430981 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:53.430981 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:53.430981 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:53.430981 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:53.430981 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:53.430981 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:53.430981 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:42:53.430981 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:42:53.430981 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:42:53.430981 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:42:53.433579 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:42:53.433579 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:42:53.433579 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:42:53.433579 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:42:53.433579 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:53.433579 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:53.433579 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:53.433579 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:53.433579 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:53.433579 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:53.433579 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:53.433579 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:53.433579 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:53.433579 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:53.433579 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:53.433579 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:53.435403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:53.435403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:53.435403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:53.435403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:53.435403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:42:53.435403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:42:53.435403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:42:53.435403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:42:53.435403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:42:53.435403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:42:53.435403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:42:53.435403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:53.436580 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49ec66-9831-4fdc-859e-b168d8831ad0 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:53.436726 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 550/1091] 10.222.0.78 () {56 vars in 1120 bytes} [Wed May 25 14:42:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:53.440133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4beda-fd86-4218-9b94-e7b6fefbaad3 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:53.442067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:53.468105 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:53.478130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099089e-9ab1-4359-945a-6e44618dfbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:53.479042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099089e-9ab1-4359-945a-6e44618dfbd6 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:53.479328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099089e-9ab1-4359-945a-6e44618dfbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:53.479598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099089e-9ab1-4359-945a-6e44618dfbd6 None admin] PATH_INFO: `/v3/users/1712d7d84c8347c78e7385fc1bea0dcb` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:53.480925 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7099089e-9ab1-4359-945a-6e44618dfbd6 None admin] RBAC: Authorizing `identity:get_user(user_id=1712d7d84c8347c78e7385fc1bea0dcb)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:53.482415 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7099089e-9ab1-4359-945a-6e44618dfbd6 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:53.484216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 542/1092] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:42:53 2022] GET /identity/v3/users/1712d7d84c8347c78e7385fc1bea0dcb => generated 285 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:53.490129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099089e-9ab1-4359-945a-6e44618dfbd6 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:53.579004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4beda-fd86-4218-9b94-e7b6fefbaad3 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:53.618790 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099089e-9ab1-4359-945a-6e44618dfbd6 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:53.621160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6d44a-eade-47cd-8b5d-290202220aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:53.621855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6d44a-eade-47cd-8b5d-290202220aa5 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:53.622183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6d44a-eade-47cd-8b5d-290202220aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:53.622461 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6d44a-eade-47cd-8b5d-290202220aa5 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:53.623248 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d6d44a-eade-47cd-8b5d-290202220aa5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:53.623555 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d6d44a-eade-47cd-8b5d-290202220aa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:53.625027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d6d44a-eade-47cd-8b5d-290202220aa5 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:53.641079 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 551/1093] 10.222.0.78 () {56 vars in 1115 bytes} [Wed May 25 14:42:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:53.642998 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6d44a-eade-47cd-8b5d-290202220aa5 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:53.672999 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a32c6-731b-4d48-9551-9355ce556aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:53.674071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a32c6-731b-4d48-9551-9355ce556aac None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:53.674475 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a32c6-731b-4d48-9551-9355ce556aac None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:53.674861 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a32c6-731b-4d48-9551-9355ce556aac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:53.737441 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:53.750581 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:53.776557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6d44a-eade-47cd-8b5d-290202220aa5 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:53.783882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a32c6-731b-4d48-9551-9355ce556aac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:53.785361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a32c6-731b-4d48-9551-9355ce556aac None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:53.829008 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d73af-86d8-47bb-bf08-826ecac888f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:53.829960 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d73af-86d8-47bb-bf08-826ecac888f8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:53.830372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d73af-86d8-47bb-bf08-826ecac888f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:53.830742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d73af-86d8-47bb-bf08-826ecac888f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:53.846111 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d73af-86d8-47bb-bf08-826ecac888f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:53.847800 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d73af-86d8-47bb-bf08-826ecac888f8 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:53.867573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-7b4d73af-86d8-47bb-bf08-826ecac888f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:53.867573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:53.867573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:53.867573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:53.867573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:53.867573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:53.867573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:53.867573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:53.867573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:53.867573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:53.867573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:53.867573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:53.867573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:53.867573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:53.867573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:53.869338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:53.869338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:53.869338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:53.869338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:53.869338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:53.869338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:53.869338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:53.869338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:53.869338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:53.869338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:53.869338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:53.869338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:53.869338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:53.869338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:53.869338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:53.869338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:53.869338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:53.870670 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:53.870670 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:53.870670 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:53.870670 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:53.870670 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 552/1094] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:42:53.877953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d73af-86d8-47bb-bf08-826ecac888f8 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:53.941774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 543/1095] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:42:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:42:53.948549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a32c6-731b-4d48-9551-9355ce556aac None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:54.011739 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d73af-86d8-47bb-bf08-826ecac888f8 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:54.046221 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:54.054467 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28438cb1-493b-441e-a837-08d36e030155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:54.055482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28438cb1-493b-441e-a837-08d36e030155 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:54.055975 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28438cb1-493b-441e-a837-08d36e030155 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:54.056437 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28438cb1-493b-441e-a837-08d36e030155 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:54.057387 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28438cb1-493b-441e-a837-08d36e030155 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:54.057946 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28438cb1-493b-441e-a837-08d36e030155 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:54.060619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28438cb1-493b-441e-a837-08d36e030155 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:54.065454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:54.083599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 553/1096] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:54.084926 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28438cb1-493b-441e-a837-08d36e030155 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:54.102474 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a32c6-731b-4d48-9551-9355ce556aac None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:54.154583 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596e10b-a5df-4613-a0b5-1dcd8c41696c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:54.155451 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596e10b-a5df-4613-a0b5-1dcd8c41696c None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:54.155730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596e10b-a5df-4613-a0b5-1dcd8c41696c None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:54.156000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596e10b-a5df-4613-a0b5-1dcd8c41696c None admin] PATH_INFO: `/v3/projects/6418c93485d7455688abb66e3431cac3` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:54.157289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6596e10b-a5df-4613-a0b5-1dcd8c41696c None admin] RBAC: Authorizing `identity:get_project(project_id=6418c93485d7455688abb66e3431cac3)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:54.159071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6596e10b-a5df-4613-a0b5-1dcd8c41696c None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:54.160905 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 544/1097] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:42:53 2022] GET /identity/v3/projects/6418c93485d7455688abb66e3431cac3 => generated 311 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:54.167810 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596e10b-a5df-4613-a0b5-1dcd8c41696c None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:54.228225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28438cb1-493b-441e-a837-08d36e030155 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:54.267728 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cce079-f961-47f5-9d76-5208750e0663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:54.268449 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cce079-f961-47f5-9d76-5208750e0663 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:54.268738 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cce079-f961-47f5-9d76-5208750e0663 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:54.269010 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cce079-f961-47f5-9d76-5208750e0663 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:54.278823 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cce079-f961-47f5-9d76-5208750e0663 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:54.280307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cce079-f961-47f5-9d76-5208750e0663 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:54.297450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-13cce079-f961-47f5-9d76-5208750e0663 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:42:54.297450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:54.297450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:54.297450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:54.297450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:54.297450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:54.297450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:54.297450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:54.297450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:54.297450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:54.297450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:54.297450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:54.297450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:42:54.297450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:42:54.297450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:42:54.297450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:42:54.304202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:54.304202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:54.304202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:42:54.304202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:54.304202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:42:54.304202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:42:54.304202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:54.304202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:54.304202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:54.304202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:54.304202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:54.304202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:54.304202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:54.304202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:54.304202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:54.304202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:54.304202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:54.304202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:54.305164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:54.305164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:54.305164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:42:54.305164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:42:54.305164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:42:54.305164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:42:54.305164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:42:54.305164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:42:54.305164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:42:54.305164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:54.305164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 554/1098] 10.222.0.78 () {56 vars in 1093 bytes} [Wed May 25 14:42:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:54.305164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cce079-f961-47f5-9d76-5208750e0663 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:54.324599 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596e10b-a5df-4613-a0b5-1dcd8c41696c None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:54.364216 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:54.384096 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889811f4-c19a-4b94-8a62-cc82a5d9a421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:54.384770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889811f4-c19a-4b94-8a62-cc82a5d9a421 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:54.385053 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889811f4-c19a-4b94-8a62-cc82a5d9a421 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:54.385362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889811f4-c19a-4b94-8a62-cc82a5d9a421 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:54.386783 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889811f4-c19a-4b94-8a62-cc82a5d9a421 None admin] RBAC: Adding query filter params (role.id=e566b37cd92f467198b5dc40f183effc, scope.project.id=6418c93485d7455688abb66e3431cac3, user.id=1712d7d84c8347c78e7385fc1bea0dcb) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:54.387095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889811f4-c19a-4b94-8a62-cc82a5d9a421 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:54.388591 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889811f4-c19a-4b94-8a62-cc82a5d9a421 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:54.395811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 545/1099] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:42:54 2022] GET /identity/v3/role_assignments?user.id=1712d7d84c8347c78e7385fc1bea0dcb&scope.project.id=6418c93485d7455688abb66e3431cac3&role.id=e566b37cd92f467198b5dc40f183effc => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:54.432974 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cce079-f961-47f5-9d76-5208750e0663 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:54.477168 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5e6d5-f7fb-475c-860f-5496c308f097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:54.477862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5e6d5-f7fb-475c-860f-5496c308f097 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:54.478186 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5e6d5-f7fb-475c-860f-5496c308f097 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:54.478482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5e6d5-f7fb-475c-860f-5496c308f097 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:54.479847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5e6d5-f7fb-475c-860f-5496c308f097 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:54.480169 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5e6d5-f7fb-475c-860f-5496c308f097 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:54.481698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5e6d5-f7fb-475c-860f-5496c308f097 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:54.511241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 555/1100] 10.222.0.78 () {56 vars in 1133 bytes} [Wed May 25 14:42:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:54.518139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889811f4-c19a-4b94-8a62-cc82a5d9a421 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:54.534728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:54.648397 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889811f4-c19a-4b94-8a62-cc82a5d9a421 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:54.688758 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af49038a-6b78-4e8f-9108-c62abf517933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:54.689432 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49038a-6b78-4e8f-9108-c62abf517933 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:54.689676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49038a-6b78-4e8f-9108-c62abf517933 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:54.689950 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49038a-6b78-4e8f-9108-c62abf517933 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:54.796432 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af49038a-6b78-4e8f-9108-c62abf517933 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:54.797789 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af49038a-6b78-4e8f-9108-c62abf517933 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:54.806558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:54.910244 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 546/1101] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:42:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:42:54.916225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5e6d5-f7fb-475c-860f-5496c308f097 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:55.060354 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5e6d5-f7fb-475c-860f-5496c308f097 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:55.072250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:55.100874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a480aa-88c3-4702-9040-e1548415fb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:55.101691 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a480aa-88c3-4702-9040-e1548415fb09 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:55.101973 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a480aa-88c3-4702-9040-e1548415fb09 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:55.102284 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a480aa-88c3-4702-9040-e1548415fb09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:55.115230 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a480aa-88c3-4702-9040-e1548415fb09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:55.116815 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a480aa-88c3-4702-9040-e1548415fb09 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:55.138542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-30a480aa-88c3-4702-9040-e1548415fb09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:55.138542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:55.138542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:55.138542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:55.138542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:55.138542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:55.138542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:55.138542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:55.138542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:55.138542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:55.138542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:55.138542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:55.138542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:42:55.138542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:42:55.138542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:42:55.140562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:42:55.140562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:55.140562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:55.140562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:55.140562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:55.140562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:55.140562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:55.140562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:55.140562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:55.140562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:55.140562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:55.140562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:55.140562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:55.140562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:55.140562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:55.140562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:55.140562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:55.142866 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:42:55.142866 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:42:55.142866 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:42:55.142866 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:55.142866 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 556/1102] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:55.148069 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af49038a-6b78-4e8f-9108-c62abf517933 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:55.329481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af49038a-6b78-4e8f-9108-c62abf517933 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:55.343528 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:55.398729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45447e-2ae1-4a6a-a35c-4e7d216d5379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:55.400016 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45447e-2ae1-4a6a-a35c-4e7d216d5379 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:55.400509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45447e-2ae1-4a6a-a35c-4e7d216d5379 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:55.401041 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45447e-2ae1-4a6a-a35c-4e7d216d5379 None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:55.402463 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45447e-2ae1-4a6a-a35c-4e7d216d5379 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:55.403030 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45447e-2ae1-4a6a-a35c-4e7d216d5379 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:55.405656 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45447e-2ae1-4a6a-a35c-4e7d216d5379 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:55.434120 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 547/1103] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:55.435263 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a480aa-88c3-4702-9040-e1548415fb09 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:55.556393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a480aa-88c3-4702-9040-e1548415fb09 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:55.622442 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa43c23-a087-45e7-b683-828c7007002a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:55.623739 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa43c23-a087-45e7-b683-828c7007002a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:55.624243 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa43c23-a087-45e7-b683-828c7007002a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:55.624746 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa43c23-a087-45e7-b683-828c7007002a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:55.638194 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa43c23-a087-45e7-b683-828c7007002a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:55.641362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa43c23-a087-45e7-b683-828c7007002a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:55.669779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-caa43c23-a087-45e7-b683-828c7007002a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:55.669779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:42:55.669779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:42:55.669779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:42:55.669779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:42:55.669779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:42:55.669779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:42:55.669779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:42:55.669779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:42:55.669779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:42:55.669779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:42:55.669779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:42:55.669779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:42:55.669779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:42:55.669779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:42:55.671587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:42:55.671587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:42:55.671587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:42:55.671587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:42:55.671587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:42:55.671587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:42:55.671587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:42:55.671587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:42:55.671587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:42:55.671587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:42:55.671587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:42:55.671587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:42:55.671587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:42:55.671587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:42:55.671587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:42:55.671587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:42:55.671587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:42:55.673391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:42:55.673391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:42:55.673391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:42:55.673391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:42:55.673391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:42:55.673391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:42:55.673391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:42:55.673391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:42:55.678287 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 557/1104] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:42:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:42:55.681715 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45447e-2ae1-4a6a-a35c-4e7d216d5379 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:55.804244 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45447e-2ae1-4a6a-a35c-4e7d216d5379 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:55.845583 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30251d64-9bc4-4266-9fdf-8947eb64cda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:55.846289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30251d64-9bc4-4266-9fdf-8947eb64cda5 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:55.846650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30251d64-9bc4-4266-9fdf-8947eb64cda5 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:55.846827 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30251d64-9bc4-4266-9fdf-8947eb64cda5 None admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:55.847621 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30251d64-9bc4-4266-9fdf-8947eb64cda5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:55.847965 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30251d64-9bc4-4266-9fdf-8947eb64cda5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:55.849391 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30251d64-9bc4-4266-9fdf-8947eb64cda5 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:55.863178 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 548/1105] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:42:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:55.870311 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa43c23-a087-45e7-b683-828c7007002a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:55.897100 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:55.995272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa43c23-a087-45e7-b683-828c7007002a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:56.035866 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4b0fa-e341-4f6e-8064-6df7c823399d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:56.036532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4b0fa-e341-4f6e-8064-6df7c823399d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:56.036815 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4b0fa-e341-4f6e-8064-6df7c823399d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:56.037087 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4b0fa-e341-4f6e-8064-6df7c823399d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:56.037928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4b0fa-e341-4f6e-8064-6df7c823399d None admin] RBAC: Adding query filter params (role.id=074ec0dc23ca41fbb4833f8857859512, scope.project.id=908a13d2b49544fbaf760b07e213bb7d, user.id=e34e1d7ffbac49b0bde45829af0c6925) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:56.038268 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4b0fa-e341-4f6e-8064-6df7c823399d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:56.039748 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4b0fa-e341-4f6e-8064-6df7c823399d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:56.046658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 558/1106] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:42:55 2022] GET /identity/v3/role_assignments?user.id=e34e1d7ffbac49b0bde45829af0c6925&scope.project.id=908a13d2b49544fbaf760b07e213bb7d&role.id=074ec0dc23ca41fbb4833f8857859512 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:56.898060 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60638b5d-fd76-43c1-bae2-59be2f01127b None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:56.898581 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60638b5d-fd76-43c1-bae2-59be2f01127b None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:56.898687 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60638b5d-fd76-43c1-bae2-59be2f01127b None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:56.900103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 549/1107] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:42:56.906954 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de328f7b-fc3f-4f7b-b267-092da064cfd8 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:56.907254 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de328f7b-fc3f-4f7b-b267-092da064cfd8 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:56.907537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de328f7b-fc3f-4f7b-b267-092da064cfd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:56.928839 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-de328f7b-fc3f-4f7b-b267-092da064cfd8 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:56.930188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-de328f7b-fc3f-4f7b-b267-092da064cfd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:57.014849 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 559/1108] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:57.056289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60638b5d-fd76-43c1-bae2-59be2f01127b None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:57.097078 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:57.191508 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60638b5d-fd76-43c1-bae2-59be2f01127b None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:57.228854 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d7ade-4a09-43b1-90af-e8e2951f1ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:57.230246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d7ade-4a09-43b1-90af-e8e2951f1ee6 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:57.230529 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d7ade-4a09-43b1-90af-e8e2951f1ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:57.230806 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d7ade-4a09-43b1-90af-e8e2951f1ee6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:57.232059 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d7ade-4a09-43b1-90af-e8e2951f1ee6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:57.233543 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d7ade-4a09-43b1-90af-e8e2951f1ee6 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:57.235330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 550/1109] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:42:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:57.241551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de328f7b-fc3f-4f7b-b267-092da064cfd8 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:57.259490 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:57.363240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de328f7b-fc3f-4f7b-b267-092da064cfd8 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:57.402186 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffec594-0bab-4bc7-9de8-0412f475f35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:57.402866 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffec594-0bab-4bc7-9de8-0412f475f35a None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:57.403147 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffec594-0bab-4bc7-9de8-0412f475f35a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:57.403419 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffec594-0bab-4bc7-9de8-0412f475f35a None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:57.404154 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffec594-0bab-4bc7-9de8-0412f475f35a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:57.405637 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffec594-0bab-4bc7-9de8-0412f475f35a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:57.429355 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 560/1110] 10.222.0.78 () {60 vars in 1132 bytes} [Wed May 25 14:42:57 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:42:59.838699 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928c7c9-0214-47fb-a194-5a4bf8bc52f2 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:59.839243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928c7c9-0214-47fb-a194-5a4bf8bc52f2 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:59.839498 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928c7c9-0214-47fb-a194-5a4bf8bc52f2 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:59.841048 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 551/1111] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:42:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:42:59.847592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11be185-2060-4ac9-bdcf-57914d365553 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:59.848016 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11be185-2060-4ac9-bdcf-57914d365553 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:59.848153 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11be185-2060-4ac9-bdcf-57914d365553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:59.864704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b11be185-2060-4ac9-bdcf-57914d365553 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:59.865903 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b11be185-2060-4ac9-bdcf-57914d365553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:59.930872 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:59.936314 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 561/1112] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:42:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:42:59.960351 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928c7c9-0214-47fb-a194-5a4bf8bc52f2 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:00.061097 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:00.069571 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928c7c9-0214-47fb-a194-5a4bf8bc52f2 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:00.106970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb84fa-dd56-4513-b436-b6797f6bd449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:00.107750 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb84fa-dd56-4513-b436-b6797f6bd449 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:00.108033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb84fa-dd56-4513-b436-b6797f6bd449 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:00.108301 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb84fa-dd56-4513-b436-b6797f6bd449 None admin] PATH_INFO: `/v3/roles/d451467ccbbf4149abbfc9b0f76b9339` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:00.109811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb84fa-dd56-4513-b436-b6797f6bd449 None admin] RBAC: Authorizing `identity:get_role(role_id=d451467ccbbf4149abbfc9b0f76b9339)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:00.111240 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb84fa-dd56-4513-b436-b6797f6bd449 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:00.112635 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 552/1113] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:42:59 2022] GET /identity/v3/roles/d451467ccbbf4149abbfc9b0f76b9339 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:00.118613 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11be185-2060-4ac9-bdcf-57914d365553 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:00.229898 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11be185-2060-4ac9-bdcf-57914d365553 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:00.267229 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf81bd5-0e43-41d5-a451-661642e6e311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:00.268285 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf81bd5-0e43-41d5-a451-661642e6e311 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:00.268654 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf81bd5-0e43-41d5-a451-661642e6e311 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:00.268961 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf81bd5-0e43-41d5-a451-661642e6e311 None admin] PATH_INFO: `/v3/users/ea9740c15f24470a903ff75b05779807` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:00.295246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf81bd5-0e43-41d5-a451-661642e6e311 None admin] RBAC: Authorizing `identity:get_user(user_id=ea9740c15f24470a903ff75b05779807)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:00.298054 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf81bd5-0e43-41d5-a451-661642e6e311 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:00.301415 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 562/1114] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:43:00 2022] GET /identity/v3/users/ea9740c15f24470a903ff75b05779807 => generated 286 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:00.308110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb84fa-dd56-4513-b436-b6797f6bd449 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:00.418407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb84fa-dd56-4513-b436-b6797f6bd449 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:00.454601 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05278587-ff6b-43b2-848c-f58e8569281f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:00.455241 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05278587-ff6b-43b2-848c-f58e8569281f None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:00.455518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05278587-ff6b-43b2-848c-f58e8569281f None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:00.455792 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05278587-ff6b-43b2-848c-f58e8569281f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:00.486852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:00.526108 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05278587-ff6b-43b2-848c-f58e8569281f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:00.527576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05278587-ff6b-43b2-848c-f58e8569281f None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:00.638029 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 553/1115] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:43:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 14:43:00.643989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf81bd5-0e43-41d5-a451-661642e6e311 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:00.705406 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:00.749688 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf81bd5-0e43-41d5-a451-661642e6e311 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:00.783803 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e497373-18d4-45dc-a0c4-730b5ac602ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:00.784655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e497373-18d4-45dc-a0c4-730b5ac602ec None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:00.784938 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e497373-18d4-45dc-a0c4-730b5ac602ec None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:00.785211 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e497373-18d4-45dc-a0c4-730b5ac602ec None admin] PATH_INFO: `/v3/projects/6418c93485d7455688abb66e3431cac3` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:00.786436 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e497373-18d4-45dc-a0c4-730b5ac602ec None admin] RBAC: Authorizing `identity:get_project(project_id=6418c93485d7455688abb66e3431cac3)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:00.787878 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e497373-18d4-45dc-a0c4-730b5ac602ec None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:00.789600 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 563/1116] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:43:00 2022] GET /identity/v3/projects/6418c93485d7455688abb66e3431cac3 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:00.795835 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05278587-ff6b-43b2-848c-f58e8569281f None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:00.858443 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:00.900282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05278587-ff6b-43b2-848c-f58e8569281f None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:00.935520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ba1b0-f37d-431e-ad19-ea8471b95190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:00.936170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ba1b0-f37d-431e-ad19-ea8471b95190 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:00.936524 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ba1b0-f37d-431e-ad19-ea8471b95190 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:00.936727 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ba1b0-f37d-431e-ad19-ea8471b95190 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:00.937556 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ba1b0-f37d-431e-ad19-ea8471b95190 None admin] RBAC: Adding query filter params (role.id=d451467ccbbf4149abbfc9b0f76b9339, scope.project.id=6418c93485d7455688abb66e3431cac3, user.id=ea9740c15f24470a903ff75b05779807) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:43:00.937866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ba1b0-f37d-431e-ad19-ea8471b95190 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:00.939338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ba1b0-f37d-431e-ad19-ea8471b95190 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:00.944816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 554/1117] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:43:00 2022] GET /identity/v3/role_assignments?user.id=ea9740c15f24470a903ff75b05779807&scope.project.id=6418c93485d7455688abb66e3431cac3&role.id=d451467ccbbf4149abbfc9b0f76b9339 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:03.168415 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34616810-3548-4128-995b-c1d7b54bc002 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:03.168945 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34616810-3548-4128-995b-c1d7b54bc002 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:03.168945 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34616810-3548-4128-995b-c1d7b54bc002 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:03.170309 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 564/1118] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:43:03.176550 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36799d13-c84e-4009-a40b-be49f13d6ac2 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:03.177094 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36799d13-c84e-4009-a40b-be49f13d6ac2 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:03.177094 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36799d13-c84e-4009-a40b-be49f13d6ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:03.195056 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-36799d13-c84e-4009-a40b-be49f13d6ac2 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:03.196266 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-36799d13-c84e-4009-a40b-be49f13d6ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:03.266550 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 555/1119] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:43:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:43:03.289427 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34616810-3548-4128-995b-c1d7b54bc002 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:03.398620 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34616810-3548-4128-995b-c1d7b54bc002 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:03.430362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:03.440833 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc4a0c-212f-4102-b7b7-d18ac4b4e876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:03.441716 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc4a0c-212f-4102-b7b7-d18ac4b4e876 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:03.441999 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc4a0c-212f-4102-b7b7-d18ac4b4e876 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:03.442303 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc4a0c-212f-4102-b7b7-d18ac4b4e876 None admin] PATH_INFO: `/v3/roles/d451467ccbbf4149abbfc9b0f76b9339` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:03.443830 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc4a0c-212f-4102-b7b7-d18ac4b4e876 None admin] RBAC: Authorizing `identity:get_role(role_id=d451467ccbbf4149abbfc9b0f76b9339)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:03.445222 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc4a0c-212f-4102-b7b7-d18ac4b4e876 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:03.446678 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 565/1120] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:43:03 2022] GET /identity/v3/roles/d451467ccbbf4149abbfc9b0f76b9339 => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:03.455163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36799d13-c84e-4009-a40b-be49f13d6ac2 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:03.512199 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:03.559262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36799d13-c84e-4009-a40b-be49f13d6ac2 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:03.593106 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd323c5-d43e-431f-923b-43315955787d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:03.593971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd323c5-d43e-431f-923b-43315955787d None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:03.594281 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd323c5-d43e-431f-923b-43315955787d None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:03.594552 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd323c5-d43e-431f-923b-43315955787d None admin] PATH_INFO: `/v3/users/ea9740c15f24470a903ff75b05779807` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:03.595752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd323c5-d43e-431f-923b-43315955787d None admin] RBAC: Authorizing `identity:get_user(user_id=ea9740c15f24470a903ff75b05779807)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:03.597198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd323c5-d43e-431f-923b-43315955787d None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:03.598890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 556/1121] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:43:03 2022] GET /identity/v3/users/ea9740c15f24470a903ff75b05779807 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:43:03.605208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc4a0c-212f-4102-b7b7-d18ac4b4e876 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:03.711126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc4a0c-212f-4102-b7b7-d18ac4b4e876 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:03.753421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344ae0b-a880-4a7e-af6d-a076625a59cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:03.754292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344ae0b-a880-4a7e-af6d-a076625a59cf None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:03.754574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344ae0b-a880-4a7e-af6d-a076625a59cf None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:03.754850 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344ae0b-a880-4a7e-af6d-a076625a59cf None admin] PATH_INFO: `/v3/projects/6418c93485d7455688abb66e3431cac3` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:03.756030 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344ae0b-a880-4a7e-af6d-a076625a59cf None admin] RBAC: Authorizing `identity:get_project(project_id=6418c93485d7455688abb66e3431cac3)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:03.757479 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344ae0b-a880-4a7e-af6d-a076625a59cf None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:03.759212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 566/1122] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:43:03 2022] GET /identity/v3/projects/6418c93485d7455688abb66e3431cac3 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:03.766450 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd323c5-d43e-431f-923b-43315955787d None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:03.867308 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd323c5-d43e-431f-923b-43315955787d None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:03.891846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:03.903435 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e417501-5d8d-477d-86a7-a4975208eeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:03.904331 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e417501-5d8d-477d-86a7-a4975208eeef None admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:03.904607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e417501-5d8d-477d-86a7-a4975208eeef None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:03.904879 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e417501-5d8d-477d-86a7-a4975208eeef None admin] PATH_INFO: `/v3/projects/6418c93485d7455688abb66e3431cac3/users/ea9740c15f24470a903ff75b05779807/roles/d451467ccbbf4149abbfc9b0f76b9339` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:03.907110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e417501-5d8d-477d-86a7-a4975208eeef None admin] RBAC: Authorizing `identity:create_grant(project_id=6418c93485d7455688abb66e3431cac3, user_id=ea9740c15f24470a903ff75b05779807, role_id=d451467ccbbf4149abbfc9b0f76b9339)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:03.908807 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e417501-5d8d-477d-86a7-a4975208eeef None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:03.921695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 557/1123] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:43:03 2022] PUT /identity/v3/projects/6418c93485d7455688abb66e3431cac3/users/ea9740c15f24470a903ff75b05779807/roles/d451467ccbbf4149abbfc9b0f76b9339 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:43:06.176475 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261cd12a-9bb6-45cf-8342-2458bbbdf4cc None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:06.177005 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261cd12a-9bb6-45cf-8342-2458bbbdf4cc None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:06.177005 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261cd12a-9bb6-45cf-8342-2458bbbdf4cc None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:06.178354 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 567/1124] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:43:06.184520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52289bf1-065c-41dd-825d-9fae5e1d6b7e None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:06.185183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52289bf1-065c-41dd-825d-9fae5e1d6b7e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:06.185183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52289bf1-065c-41dd-825d-9fae5e1d6b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:06.203451 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-52289bf1-065c-41dd-825d-9fae5e1d6b7e None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:06.204798 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-52289bf1-065c-41dd-825d-9fae5e1d6b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:06.278934 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 558/1125] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:43:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:43:06.303413 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261cd12a-9bb6-45cf-8342-2458bbbdf4cc None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:06.382313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:06.422589 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261cd12a-9bb6-45cf-8342-2458bbbdf4cc None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:06.461415 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77201011-09b5-454d-b3d6-bdf2e15d18b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:06.462209 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77201011-09b5-454d-b3d6-bdf2e15d18b3 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:06.462488 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77201011-09b5-454d-b3d6-bdf2e15d18b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:06.462816 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77201011-09b5-454d-b3d6-bdf2e15d18b3 None admin] PATH_INFO: `/v3/roles/d451467ccbbf4149abbfc9b0f76b9339` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:06.464315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77201011-09b5-454d-b3d6-bdf2e15d18b3 None admin] RBAC: Authorizing `identity:get_role(role_id=d451467ccbbf4149abbfc9b0f76b9339)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:06.465677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77201011-09b5-454d-b3d6-bdf2e15d18b3 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:06.467085 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 568/1126] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:43:06 2022] GET /identity/v3/roles/d451467ccbbf4149abbfc9b0f76b9339 => generated 223 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:06.472959 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52289bf1-065c-41dd-825d-9fae5e1d6b7e None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:06.580083 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52289bf1-065c-41dd-825d-9fae5e1d6b7e None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:06.596587 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:06.620161 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d169f5d-00a2-4951-a076-d647a894b2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:06.621040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d169f5d-00a2-4951-a076-d647a894b2b4 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:06.621323 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d169f5d-00a2-4951-a076-d647a894b2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:06.621598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d169f5d-00a2-4951-a076-d647a894b2b4 None admin] PATH_INFO: `/v3/users/ea9740c15f24470a903ff75b05779807` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:06.622836 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d169f5d-00a2-4951-a076-d647a894b2b4 None admin] RBAC: Authorizing `identity:get_user(user_id=ea9740c15f24470a903ff75b05779807)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:06.624267 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d169f5d-00a2-4951-a076-d647a894b2b4 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:06.625924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 559/1127] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:43:06 2022] GET /identity/v3/users/ea9740c15f24470a903ff75b05779807 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:06.632090 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77201011-09b5-454d-b3d6-bdf2e15d18b3 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:06.736540 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77201011-09b5-454d-b3d6-bdf2e15d18b3 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:06.765817 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:06.772952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c899a-df1e-4d4c-9a2f-9a789e1e1e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:06.773619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c899a-df1e-4d4c-9a2f-9a789e1e1e7b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:06.773901 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c899a-df1e-4d4c-9a2f-9a789e1e1e7b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:06.774219 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c899a-df1e-4d4c-9a2f-9a789e1e1e7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:06.853268 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c899a-df1e-4d4c-9a2f-9a789e1e1e7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:06.854798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c899a-df1e-4d4c-9a2f-9a789e1e1e7b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:06.964689 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 569/1128] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:43:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:43:06.971254 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d169f5d-00a2-4951-a076-d647a894b2b4 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:07.079184 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d169f5d-00a2-4951-a076-d647a894b2b4 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:07.117882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281883e-c7e1-453d-aa9b-490c1f59ee9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:07.118820 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281883e-c7e1-453d-aa9b-490c1f59ee9a None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:07.119108 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281883e-c7e1-453d-aa9b-490c1f59ee9a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:07.119385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281883e-c7e1-453d-aa9b-490c1f59ee9a None admin] PATH_INFO: `/v3/projects/6418c93485d7455688abb66e3431cac3` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:07.120691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d281883e-c7e1-453d-aa9b-490c1f59ee9a None admin] RBAC: Authorizing `identity:get_project(project_id=6418c93485d7455688abb66e3431cac3)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:07.122209 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d281883e-c7e1-453d-aa9b-490c1f59ee9a None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:07.124017 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 560/1129] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:43:06 2022] GET /identity/v3/projects/6418c93485d7455688abb66e3431cac3 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:07.130761 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c899a-df1e-4d4c-9a2f-9a789e1e1e7b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:07.162745 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:07.238733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c899a-df1e-4d4c-9a2f-9a789e1e1e7b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:07.274225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baffea3f-d26b-480c-900c-3152444226a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:07.274886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffea3f-d26b-480c-900c-3152444226a8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:07.275238 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffea3f-d26b-480c-900c-3152444226a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:07.276029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffea3f-d26b-480c-900c-3152444226a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:07.276866 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baffea3f-d26b-480c-900c-3152444226a8 None admin] RBAC: Adding query filter params (role.id=d451467ccbbf4149abbfc9b0f76b9339, scope.project.id=6418c93485d7455688abb66e3431cac3, user.id=ea9740c15f24470a903ff75b05779807) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:43:07.277188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baffea3f-d26b-480c-900c-3152444226a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:07.278681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baffea3f-d26b-480c-900c-3152444226a8 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:07.285264 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 570/1130] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:43:07 2022] GET /identity/v3/role_assignments?user.id=ea9740c15f24470a903ff75b05779807&scope.project.id=6418c93485d7455688abb66e3431cac3&role.id=d451467ccbbf4149abbfc9b0f76b9339 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:43:09.609804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06316a5a-3212-4b6e-a5ac-884e76a5f513 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:09.610545 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06316a5a-3212-4b6e-a5ac-884e76a5f513 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:09.611007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06316a5a-3212-4b6e-a5ac-884e76a5f513 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:09.612987 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 561/1131] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:43:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:43:09.619546 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c80fc-a12e-415d-ac05-3dd408f6bc09 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:09.620163 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c80fc-a12e-415d-ac05-3dd408f6bc09 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:09.620163 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c80fc-a12e-415d-ac05-3dd408f6bc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:09.638263 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-711c80fc-a12e-415d-ac05-3dd408f6bc09 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:09.639474 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-711c80fc-a12e-415d-ac05-3dd408f6bc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:09.707345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 571/1132] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:43:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:43:09.730590 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06316a5a-3212-4b6e-a5ac-884e76a5f513 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:09.781998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:09.834825 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06316a5a-3212-4b6e-a5ac-884e76a5f513 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:09.871746 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142181a-109f-4f3e-833a-a6380ab745e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:09.872467 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142181a-109f-4f3e-833a-a6380ab745e1 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:09.872746 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142181a-109f-4f3e-833a-a6380ab745e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:09.873012 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142181a-109f-4f3e-833a-a6380ab745e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:09.874306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142181a-109f-4f3e-833a-a6380ab745e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:09.875773 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142181a-109f-4f3e-833a-a6380ab745e1 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:09.877522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 562/1133] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:43:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:09.883796 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c80fc-a12e-415d-ac05-3dd408f6bc09 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:09.911938 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:09.986896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c80fc-a12e-415d-ac05-3dd408f6bc09 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:10.021268 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec4e1a-efb2-4d84-b839-4192fa7650a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:10.021933 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec4e1a-efb2-4d84-b839-4192fa7650a0 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:10.022247 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec4e1a-efb2-4d84-b839-4192fa7650a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:10.022528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec4e1a-efb2-4d84-b839-4192fa7650a0 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:10.023235 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec4e1a-efb2-4d84-b839-4192fa7650a0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:10.024691 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec4e1a-efb2-4d84-b839-4192fa7650a0 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:10.051587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 572/1134] 10.222.0.78 () {60 vars in 1141 bytes} [Wed May 25 14:43:09 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:43:12.322721 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb0a3b-e0b7-4fee-bc06-17ae7efcd412 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:12.323284 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb0a3b-e0b7-4fee-bc06-17ae7efcd412 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:12.323284 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb0a3b-e0b7-4fee-bc06-17ae7efcd412 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:12.324643 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 563/1135] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:43:12.330024 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c686e1-88dc-4312-b724-b85d73386367 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:12.330434 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c686e1-88dc-4312-b724-b85d73386367 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:12.330597 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c686e1-88dc-4312-b724-b85d73386367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:12.347703 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b2c686e1-88dc-4312-b724-b85d73386367 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:12.348929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b2c686e1-88dc-4312-b724-b85d73386367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:12.418364 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 573/1136] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:43:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:43:12.443218 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb0a3b-e0b7-4fee-bc06-17ae7efcd412 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:12.556058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb0a3b-e0b7-4fee-bc06-17ae7efcd412 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:12.575504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:12.593303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3bb2c-530e-47e3-8d09-56fe2ad329a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:12.594136 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3bb2c-530e-47e3-8d09-56fe2ad329a7 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:12.594419 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3bb2c-530e-47e3-8d09-56fe2ad329a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:12.594688 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3bb2c-530e-47e3-8d09-56fe2ad329a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:12.595902 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3bb2c-530e-47e3-8d09-56fe2ad329a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:12.597337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3bb2c-530e-47e3-8d09-56fe2ad329a7 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:12.599026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 564/1137] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:43:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:12.605072 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c686e1-88dc-4312-b724-b85d73386367 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:12.614055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:12.707156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c686e1-88dc-4312-b724-b85d73386367 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:12.743260 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5858e3-42a1-4f08-ba94-d5a4a242720f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:12.743968 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5858e3-42a1-4f08-ba94-d5a4a242720f None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:12.744255 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5858e3-42a1-4f08-ba94-d5a4a242720f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:12.744534 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5858e3-42a1-4f08-ba94-d5a4a242720f None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:12.745276 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5858e3-42a1-4f08-ba94-d5a4a242720f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:12.746756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5858e3-42a1-4f08-ba94-d5a4a242720f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:12.768261 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 574/1138] 10.222.0.78 () {60 vars in 1132 bytes} [Wed May 25 14:43:12 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:43:15.061077 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277acfac-1740-4506-a7ee-71d88d5cd888 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:15.061867 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277acfac-1740-4506-a7ee-71d88d5cd888 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:15.061867 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277acfac-1740-4506-a7ee-71d88d5cd888 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:15.063694 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 565/1139] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:43:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:43:15.071295 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d0578-93c5-4392-a995-22ca8bf28b29 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:15.072289 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d0578-93c5-4392-a995-22ca8bf28b29 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:15.072398 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d0578-93c5-4392-a995-22ca8bf28b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:15.089316 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-155d0578-93c5-4392-a995-22ca8bf28b29 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:15.090519 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-155d0578-93c5-4392-a995-22ca8bf28b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:15.148927 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:15.161018 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 575/1140] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:43:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:43:15.184631 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277acfac-1740-4506-a7ee-71d88d5cd888 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:15.291447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277acfac-1740-4506-a7ee-71d88d5cd888 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:15.328264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0315a-fb58-4380-a5d5-18b8f977d23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:15.329049 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0315a-fb58-4380-a5d5-18b8f977d23a None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:15.329324 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0315a-fb58-4380-a5d5-18b8f977d23a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:15.329589 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0315a-fb58-4380-a5d5-18b8f977d23a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:15.340194 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0315a-fb58-4380-a5d5-18b8f977d23a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:43:15.347776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-04d0315a-fb58-4380-a5d5-18b8f977d23a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:43:15.347776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:43:15.347776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:43:15.347776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:43:15.347776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:43:15.347776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:43:15.347776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:43:15.347776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:43:15.347776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:43:15.347776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:43:15.347776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:43:15.347776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:43:15.347776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:43:15.347776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:43:15.347776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:43:15.347776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:43:15.348920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:43:15.348920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:43:15.348920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:43:15.348920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:43:15.348920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:43:15.348920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:43:15.348920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:43:15.348920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:43:15.348920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:43:15.348920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:43:15.348920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:43:15.348920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:43:15.348920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:43:15.348920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:43:15.348920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:43:15.348920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:43:15.349824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:43:15.349824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:43:15.349824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:43:15.349824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:43:15.349824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:43:15.349824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:43:15.349824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:43:15.349824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:43:15.349824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:43:15.349824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:43:15.349824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:43:15.349824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:43:15.349824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 566/1141] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:43:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:43:15.358060 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d0578-93c5-4392-a995-22ca8bf28b29 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:15.468444 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d0578-93c5-4392-a995-22ca8bf28b29 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:15.502421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f022af3-3ddd-4e61-a536-e6be3eaf3546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:15.503089 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f022af3-3ddd-4e61-a536-e6be3eaf3546 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:15.503372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f022af3-3ddd-4e61-a536-e6be3eaf3546 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:15.503649 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f022af3-3ddd-4e61-a536-e6be3eaf3546 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:15.504395 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f022af3-3ddd-4e61-a536-e6be3eaf3546 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:43:15.504700 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f022af3-3ddd-4e61-a536-e6be3eaf3546 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:15.506092 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f022af3-3ddd-4e61-a536-e6be3eaf3546 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:15.515098 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 576/1142] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:43:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:15.521183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0315a-fb58-4380-a5d5-18b8f977d23a None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:15.574961 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:15.631470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0315a-fb58-4380-a5d5-18b8f977d23a None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:15.669192 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8b152-1547-4d89-ad77-6ec3b4e092af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:15.670078 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8b152-1547-4d89-ad77-6ec3b4e092af None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:15.670358 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8b152-1547-4d89-ad77-6ec3b4e092af None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:15.670628 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8b152-1547-4d89-ad77-6ec3b4e092af None admin] PATH_INFO: `/v3/users/4b10c120e66e4f11b0057e4344cfe305` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:15.686121 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8b152-1547-4d89-ad77-6ec3b4e092af None admin] RBAC: Authorizing `identity:get_user(user_id=4b10c120e66e4f11b0057e4344cfe305)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:15.687702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8b152-1547-4d89-ad77-6ec3b4e092af None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:15.689467 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 567/1143] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:43:15 2022] GET /identity/v3/users/4b10c120e66e4f11b0057e4344cfe305 => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:15.695553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f022af3-3ddd-4e61-a536-e6be3eaf3546 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:15.751399 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:15.796754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f022af3-3ddd-4e61-a536-e6be3eaf3546 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:15.831433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a436fb-d66f-458c-8cd3-0ed2966ee4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:15.832089 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a436fb-d66f-458c-8cd3-0ed2966ee4a2 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:15.832372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a436fb-d66f-458c-8cd3-0ed2966ee4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:15.832644 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a436fb-d66f-458c-8cd3-0ed2966ee4a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:15.910561 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a436fb-d66f-458c-8cd3-0ed2966ee4a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:15.912051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a436fb-d66f-458c-8cd3-0ed2966ee4a2 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:15.968437 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:16.015933 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 577/1144] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:43:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:43:16.021849 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8b152-1547-4d89-ad77-6ec3b4e092af None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:16.126987 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8b152-1547-4d89-ad77-6ec3b4e092af None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:16.129371 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:16.167728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2187440-0ace-4585-b659-ec59159e5884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:16.168610 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2187440-0ace-4585-b659-ec59159e5884 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:16.168886 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2187440-0ace-4585-b659-ec59159e5884 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:16.169158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2187440-0ace-4585-b659-ec59159e5884 None admin] PATH_INFO: `/v3/projects/e9b01ef4220d40e8859a063f8d9007c1` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:16.170444 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2187440-0ace-4585-b659-ec59159e5884 None admin] RBAC: Authorizing `identity:get_project(project_id=e9b01ef4220d40e8859a063f8d9007c1)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:16.171909 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2187440-0ace-4585-b659-ec59159e5884 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:16.173671 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 568/1145] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:43:16 2022] GET /identity/v3/projects/e9b01ef4220d40e8859a063f8d9007c1 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:16.179825 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a436fb-d66f-458c-8cd3-0ed2966ee4a2 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:16.278365 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a436fb-d66f-458c-8cd3-0ed2966ee4a2 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:16.312834 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d105dd-a889-48fa-a2d9-d581fcdabce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:16.313490 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d105dd-a889-48fa-a2d9-d581fcdabce3 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:16.313775 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d105dd-a889-48fa-a2d9-d581fcdabce3 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:16.314078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d105dd-a889-48fa-a2d9-d581fcdabce3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:16.314933 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d105dd-a889-48fa-a2d9-d581fcdabce3 None admin] RBAC: Adding query filter params (role.id=e566b37cd92f467198b5dc40f183effc, scope.project.id=e9b01ef4220d40e8859a063f8d9007c1, user.id=4b10c120e66e4f11b0057e4344cfe305) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:43:16.315247 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d105dd-a889-48fa-a2d9-d581fcdabce3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:16.316700 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d105dd-a889-48fa-a2d9-d581fcdabce3 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:16.322840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 578/1146] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:43:16 2022] GET /identity/v3/role_assignments?user.id=4b10c120e66e4f11b0057e4344cfe305&scope.project.id=e9b01ef4220d40e8859a063f8d9007c1&role.id=e566b37cd92f467198b5dc40f183effc => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:18.519821 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cd0e4-98d3-4b7b-8c49-07fd611359e3 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:18.520353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cd0e4-98d3-4b7b-8c49-07fd611359e3 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:18.520353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cd0e4-98d3-4b7b-8c49-07fd611359e3 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:18.521724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 569/1147] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:43:18.527065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b5290-e8b3-439a-b478-624a032d5dce None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:18.527499 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b5290-e8b3-439a-b478-624a032d5dce None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:18.527626 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b5290-e8b3-439a-b478-624a032d5dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:18.545872 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-bd5b5290-e8b3-439a-b478-624a032d5dce None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:18.547125 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-bd5b5290-e8b3-439a-b478-624a032d5dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:18.561345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:18.616515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 579/1148] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:43:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:43:18.639500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cd0e4-98d3-4b7b-8c49-07fd611359e3 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:18.744110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cd0e4-98d3-4b7b-8c49-07fd611359e3 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:18.779637 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ba204-b14a-475f-bcc8-4ed2c122fb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:18.780350 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ba204-b14a-475f-bcc8-4ed2c122fb0c None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:18.780631 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ba204-b14a-475f-bcc8-4ed2c122fb0c None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:18.780906 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ba204-b14a-475f-bcc8-4ed2c122fb0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:18.791689 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ba204-b14a-475f-bcc8-4ed2c122fb0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:43:18.799288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-453ba204-b14a-475f-bcc8-4ed2c122fb0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:43:18.799288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:43:18.799288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:43:18.799288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:43:18.799288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:43:18.799288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:43:18.799288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:43:18.799288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:43:18.799288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:43:18.799288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:43:18.799288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:43:18.799288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:43:18.799288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:43:18.799288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:43:18.799288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:43:18.799288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:43:18.800407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:43:18.800407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:43:18.800407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:43:18.800407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:43:18.800407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:43:18.800407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:43:18.800407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:43:18.800407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:43:18.800407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:43:18.800407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:43:18.800407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:43:18.800407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:43:18.800407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:43:18.800407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:43:18.800407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:43:18.800407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:43:18.801283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:43:18.801283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:43:18.801283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:43:18.801283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:43:18.801283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:43:18.801283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:43:18.801283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:43:18.801283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:43:18.801283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:43:18.801283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:43:18.801283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:43:18.801283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:43:18.801283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 570/1149] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:43:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:43:18.806609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b5290-e8b3-439a-b478-624a032d5dce None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:18.910322 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b5290-e8b3-439a-b478-624a032d5dce None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:18.951937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600fb7d-cb1f-4b83-84b3-a2c8c2fb04f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:18.952637 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600fb7d-cb1f-4b83-84b3-a2c8c2fb04f6 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:18.952922 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600fb7d-cb1f-4b83-84b3-a2c8c2fb04f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:18.953201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600fb7d-cb1f-4b83-84b3-a2c8c2fb04f6 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:18.953984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600fb7d-cb1f-4b83-84b3-a2c8c2fb04f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:43:18.954454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600fb7d-cb1f-4b83-84b3-a2c8c2fb04f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:18.956890 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600fb7d-cb1f-4b83-84b3-a2c8c2fb04f6 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:18.970866 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 580/1150] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:43:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:18.976997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ba204-b14a-475f-bcc8-4ed2c122fb0c None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:19.009823 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:19.079770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ba204-b14a-475f-bcc8-4ed2c122fb0c None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:19.115817 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c1887-463d-468e-a8fa-2ef7e5411146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:19.116680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c1887-463d-468e-a8fa-2ef7e5411146 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:19.116960 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c1887-463d-468e-a8fa-2ef7e5411146 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:19.117233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c1887-463d-468e-a8fa-2ef7e5411146 None admin] PATH_INFO: `/v3/users/4b10c120e66e4f11b0057e4344cfe305` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:19.118484 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c1887-463d-468e-a8fa-2ef7e5411146 None admin] RBAC: Authorizing `identity:get_user(user_id=4b10c120e66e4f11b0057e4344cfe305)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:19.119924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c1887-463d-468e-a8fa-2ef7e5411146 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:19.121580 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 571/1151] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:43:18 2022] GET /identity/v3/users/4b10c120e66e4f11b0057e4344cfe305 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:19.129042 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600fb7d-cb1f-4b83-84b3-a2c8c2fb04f6 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:19.143477 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:19.233504 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600fb7d-cb1f-4b83-84b3-a2c8c2fb04f6 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:19.269263 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c21dd4d-2c72-4d9d-9721-6d8df2d4e054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:19.270648 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21dd4d-2c72-4d9d-9721-6d8df2d4e054 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:19.270778 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21dd4d-2c72-4d9d-9721-6d8df2d4e054 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:19.273826 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21dd4d-2c72-4d9d-9721-6d8df2d4e054 None admin] PATH_INFO: `/v3/projects/e9b01ef4220d40e8859a063f8d9007c1` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:19.273826 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c21dd4d-2c72-4d9d-9721-6d8df2d4e054 None admin] RBAC: Authorizing `identity:get_project(project_id=e9b01ef4220d40e8859a063f8d9007c1)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:19.274316 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c21dd4d-2c72-4d9d-9721-6d8df2d4e054 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:19.277362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 581/1152] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:43:19 2022] GET /identity/v3/projects/e9b01ef4220d40e8859a063f8d9007c1 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:19.284731 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c1887-463d-468e-a8fa-2ef7e5411146 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:19.389843 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:19.392909 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c1887-463d-468e-a8fa-2ef7e5411146 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:19.431494 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d6cb8-ac0a-4780-af2e-046b5f255baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:19.432386 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d6cb8-ac0a-4780-af2e-046b5f255baa None admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:19.432666 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d6cb8-ac0a-4780-af2e-046b5f255baa None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:19.432941 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d6cb8-ac0a-4780-af2e-046b5f255baa None admin] PATH_INFO: `/v3/projects/e9b01ef4220d40e8859a063f8d9007c1/users/4b10c120e66e4f11b0057e4344cfe305/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:19.434964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d6cb8-ac0a-4780-af2e-046b5f255baa None admin] RBAC: Authorizing `identity:create_grant(project_id=e9b01ef4220d40e8859a063f8d9007c1, user_id=4b10c120e66e4f11b0057e4344cfe305, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:19.436392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d6cb8-ac0a-4780-af2e-046b5f255baa None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:19.451220 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 572/1153] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:43:19 2022] PUT /identity/v3/projects/e9b01ef4220d40e8859a063f8d9007c1/users/4b10c120e66e4f11b0057e4344cfe305/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:43:21.714730 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f54ea2-9dd4-4d03-b10f-3f1895f3b8ae None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:21.715307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f54ea2-9dd4-4d03-b10f-3f1895f3b8ae None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:21.715307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f54ea2-9dd4-4d03-b10f-3f1895f3b8ae None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:21.716634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 582/1154] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:43:21.721851 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bdc28c-7b00-4697-b255-e45820f721cb None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:21.722326 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bdc28c-7b00-4697-b255-e45820f721cb None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:21.722425 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bdc28c-7b00-4697-b255-e45820f721cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:21.739014 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-78bdc28c-7b00-4697-b255-e45820f721cb None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:21.740195 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-78bdc28c-7b00-4697-b255-e45820f721cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:21.810974 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 573/1155] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:43:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:43:21.834487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f54ea2-9dd4-4d03-b10f-3f1895f3b8ae None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:21.916765 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:21.940292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f54ea2-9dd4-4d03-b10f-3f1895f3b8ae None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:21.976731 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb197f-32b8-41c9-b502-eafcebf1da16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:21.977467 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb197f-32b8-41c9-b502-eafcebf1da16 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:21.977753 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb197f-32b8-41c9-b502-eafcebf1da16 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:21.978055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb197f-32b8-41c9-b502-eafcebf1da16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:21.988806 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb197f-32b8-41c9-b502-eafcebf1da16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:43:21.996917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-6cdb197f-32b8-41c9-b502-eafcebf1da16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:43:21.996917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:43:21.996917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:43:21.996917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:43:21.996917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:43:21.996917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:43:21.996917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:43:21.996917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:43:21.996917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:43:21.996917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:43:21.996917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:43:21.996917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:43:21.996917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:43:21.996917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:43:21.996917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:43:21.996917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:43:21.998171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:43:21.998171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:43:21.998171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:43:21.998171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:43:21.998171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:43:21.998171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:43:21.998171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:43:21.998171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:43:21.998171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:43:21.998171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:43:21.998171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:43:21.998171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:43:21.998171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:43:21.998171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:43:21.998171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:43:21.998171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:43:21.999123 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:43:21.999123 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:43:21.999123 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:43:21.999123 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:43:21.999123 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:43:21.999123 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:43:21.999123 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:43:21.999123 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:43:21.999123 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:43:21.999123 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:43:21.999123 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:43:21.999123 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:43:21.999123 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 583/1156] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:43:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:43:22.004512 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bdc28c-7b00-4697-b255-e45820f721cb None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:22.098270 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:22.110670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bdc28c-7b00-4697-b255-e45820f721cb None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:22.148370 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8bfb8-89d3-48bb-b28b-3d909a6438ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:22.149043 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8bfb8-89d3-48bb-b28b-3d909a6438ab None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:22.149858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8bfb8-89d3-48bb-b28b-3d909a6438ab None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:22.150169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8bfb8-89d3-48bb-b28b-3d909a6438ab None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:22.150926 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8bfb8-89d3-48bb-b28b-3d909a6438ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:43:22.151231 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8bfb8-89d3-48bb-b28b-3d909a6438ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:22.152606 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8bfb8-89d3-48bb-b28b-3d909a6438ab None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:22.161490 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 574/1157] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:43:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:22.168024 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb197f-32b8-41c9-b502-eafcebf1da16 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:22.274155 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb197f-32b8-41c9-b502-eafcebf1da16 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:22.309344 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963282bb-4442-4ddf-8757-fb74197d2094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:22.310267 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963282bb-4442-4ddf-8757-fb74197d2094 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:22.310555 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963282bb-4442-4ddf-8757-fb74197d2094 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:22.310831 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963282bb-4442-4ddf-8757-fb74197d2094 None admin] PATH_INFO: `/v3/users/4b10c120e66e4f11b0057e4344cfe305` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:22.312083 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963282bb-4442-4ddf-8757-fb74197d2094 None admin] RBAC: Authorizing `identity:get_user(user_id=4b10c120e66e4f11b0057e4344cfe305)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:22.313553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963282bb-4442-4ddf-8757-fb74197d2094 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:22.315296 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 584/1158] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:43:22 2022] GET /identity/v3/users/4b10c120e66e4f11b0057e4344cfe305 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:43:22.321842 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8bfb8-89d3-48bb-b28b-3d909a6438ab None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:22.435976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8bfb8-89d3-48bb-b28b-3d909a6438ab None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:22.471065 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3d903-870c-4c60-a5b6-df75b778a8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:22.471711 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3d903-870c-4c60-a5b6-df75b778a8ac None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:22.471994 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3d903-870c-4c60-a5b6-df75b778a8ac None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:22.472270 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3d903-870c-4c60-a5b6-df75b778a8ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:22.496376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:22.544108 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3d903-870c-4c60-a5b6-df75b778a8ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:22.545507 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3d903-870c-4c60-a5b6-df75b778a8ac None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:22.647533 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 575/1159] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:43:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:43:22.653533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963282bb-4442-4ddf-8757-fb74197d2094 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:22.662836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:22.753310 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963282bb-4442-4ddf-8757-fb74197d2094 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:22.788686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce53e92-71fe-4956-a7a5-58c2c94309fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:22.789541 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce53e92-71fe-4956-a7a5-58c2c94309fd None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:22.789858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce53e92-71fe-4956-a7a5-58c2c94309fd None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:22.790166 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce53e92-71fe-4956-a7a5-58c2c94309fd None admin] PATH_INFO: `/v3/projects/e9b01ef4220d40e8859a063f8d9007c1` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:22.791343 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce53e92-71fe-4956-a7a5-58c2c94309fd None admin] RBAC: Authorizing `identity:get_project(project_id=e9b01ef4220d40e8859a063f8d9007c1)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:22.792802 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce53e92-71fe-4956-a7a5-58c2c94309fd None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:22.794591 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 585/1160] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:43:22 2022] GET /identity/v3/projects/e9b01ef4220d40e8859a063f8d9007c1 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:22.800613 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3d903-870c-4c60-a5b6-df75b778a8ac None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:22.859602 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:22.908581 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3d903-870c-4c60-a5b6-df75b778a8ac None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:22.943829 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8108aa-54c3-4230-b6ee-21b08ebc3636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:22.944477 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8108aa-54c3-4230-b6ee-21b08ebc3636 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:22.944752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8108aa-54c3-4230-b6ee-21b08ebc3636 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:22.945020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8108aa-54c3-4230-b6ee-21b08ebc3636 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:22.945897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8108aa-54c3-4230-b6ee-21b08ebc3636 None admin] RBAC: Adding query filter params (role.id=e566b37cd92f467198b5dc40f183effc, scope.project.id=e9b01ef4220d40e8859a063f8d9007c1, user.id=4b10c120e66e4f11b0057e4344cfe305) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:43:22.946261 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8108aa-54c3-4230-b6ee-21b08ebc3636 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:22.947680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8108aa-54c3-4230-b6ee-21b08ebc3636 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:22.953445 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 576/1161] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:43:22 2022] GET /identity/v3/role_assignments?user.id=4b10c120e66e4f11b0057e4344cfe305&scope.project.id=e9b01ef4220d40e8859a063f8d9007c1&role.id=e566b37cd92f467198b5dc40f183effc => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:25.209421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b7a80-cdc5-44ad-82d3-8c4868f25f81 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:25.209934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b7a80-cdc5-44ad-82d3-8c4868f25f81 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:25.209934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b7a80-cdc5-44ad-82d3-8c4868f25f81 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:25.211337 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 586/1162] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:43:25.217762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7aa3f-49c4-459a-a5d6-733489f1d5fb None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:25.218314 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7aa3f-49c4-459a-a5d6-733489f1d5fb None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:25.218413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7aa3f-49c4-459a-a5d6-733489f1d5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:25.236076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-d6c7aa3f-49c4-459a-a5d6-733489f1d5fb None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:25.237279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d6c7aa3f-49c4-459a-a5d6-733489f1d5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:25.307259 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 577/1163] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:43:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:43:25.330064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b7a80-cdc5-44ad-82d3-8c4868f25f81 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:25.412366 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:25.444121 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b7a80-cdc5-44ad-82d3-8c4868f25f81 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:25.481384 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a82bd-5d3d-4286-a9ec-64cad70983d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:25.482057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a82bd-5d3d-4286-a9ec-64cad70983d4 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:25.482342 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a82bd-5d3d-4286-a9ec-64cad70983d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:25.482612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a82bd-5d3d-4286-a9ec-64cad70983d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:25.552371 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a82bd-5d3d-4286-a9ec-64cad70983d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:25.553825 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a82bd-5d3d-4286-a9ec-64cad70983d4 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:25.633491 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:25.665388 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 587/1164] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:43:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:43:25.671562 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7aa3f-49c4-459a-a5d6-733489f1d5fb None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:25.717622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:25.769920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7aa3f-49c4-459a-a5d6-733489f1d5fb None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:25.803174 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a87f21-6ce0-4016-bf22-04742f61d2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:25.803972 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a87f21-6ce0-4016-bf22-04742f61d2ec None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:25.804253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a87f21-6ce0-4016-bf22-04742f61d2ec None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:25.804524 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a87f21-6ce0-4016-bf22-04742f61d2ec None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:25.811704 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a87f21-6ce0-4016-bf22-04742f61d2ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:25.813139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a87f21-6ce0-4016-bf22-04742f61d2ec None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:25.827623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-f6a87f21-6ce0-4016-bf22-04742f61d2ec None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 25 14:43:25.827623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:43:25.827623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 25 14:43:25.827623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 25 14:43:25.827623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:43:25.827623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:43:25.827623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:43:25.827623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:43:25.827623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 25 14:43:25.827623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:43:25.827623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:43:25.827623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:43:25.827623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:43:25.827623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:43:25.827623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:43:25.827623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:43:25.827623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:43:25.828866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:43:25.828866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:43:25.828866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:43:25.828866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:43:25.828866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:43:25.828866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:43:25.828866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:43:25.828866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:43:25.828866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:43:25.828866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:43:25.828866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:43:25.828866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:43:25.828866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:43:25.828866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:43:25.828866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:43:25.828866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:43:25.829764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:43:25.829764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:43:25.829764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:43:25.829764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:43:25.829764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:43:25.829764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:43:25.829764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:43:25.829764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:43:25.829764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:43:25.829764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:43:25.829764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 25 14:43:25.829764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:43:25.829764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 25 14:43:25.829764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:43:25.829764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 578/1165] 10.222.0.78 () {56 vars in 1117 bytes} [Wed May 25 14:43:25 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:43:25.834911 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a82bd-5d3d-4286-a9ec-64cad70983d4 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:25.933482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a82bd-5d3d-4286-a9ec-64cad70983d4 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:25.967686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f17265-4262-4ad2-ad05-89eeab951e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:25.968343 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f17265-4262-4ad2-ad05-89eeab951e85 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:25.968644 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f17265-4262-4ad2-ad05-89eeab951e85 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:25.968917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f17265-4262-4ad2-ad05-89eeab951e85 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:25.969637 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f17265-4262-4ad2-ad05-89eeab951e85 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:43:25.969937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f17265-4262-4ad2-ad05-89eeab951e85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:25.971335 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f17265-4262-4ad2-ad05-89eeab951e85 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:25.979591 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 588/1166] 10.222.0.78 () {56 vars in 1115 bytes} [Wed May 25 14:43:25 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:28.188874 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e194bc-a61d-42be-9508-3b7e78497905 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:28.189387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e194bc-a61d-42be-9508-3b7e78497905 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:28.189387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e194bc-a61d-42be-9508-3b7e78497905 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:28.190738 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 579/1167] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:43:28.195965 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792ebb1-420f-4fa6-9bda-30a6403d3d5c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:28.196416 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792ebb1-420f-4fa6-9bda-30a6403d3d5c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:28.196516 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792ebb1-420f-4fa6-9bda-30a6403d3d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:28.213409 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b792ebb1-420f-4fa6-9bda-30a6403d3d5c None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:28.214622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b792ebb1-420f-4fa6-9bda-30a6403d3d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:28.243633 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:28.281271 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 589/1168] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:43:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:43:28.303750 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e194bc-a61d-42be-9508-3b7e78497905 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:28.409778 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e194bc-a61d-42be-9508-3b7e78497905 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:28.420714 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:28.451844 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62715ae-8b7b-43d0-8bc4-2410d2a31180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:28.452419 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62715ae-8b7b-43d0-8bc4-2410d2a31180 None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:28.452699 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62715ae-8b7b-43d0-8bc4-2410d2a31180 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:28.452975 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62715ae-8b7b-43d0-8bc4-2410d2a31180 None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:28.453721 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62715ae-8b7b-43d0-8bc4-2410d2a31180 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:28.455119 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62715ae-8b7b-43d0-8bc4-2410d2a31180 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:28.469916 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 580/1169] 10.222.0.78 () {60 vars in 1138 bytes} [Wed May 25 14:43:28 2022] POST /identity/v3/domains => generated 267 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:43:30.709530 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e277b0-9628-401e-8697-84707c6af6fa None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:30.710126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e277b0-9628-401e-8697-84707c6af6fa None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:30.710126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e277b0-9628-401e-8697-84707c6af6fa None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:30.712077 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 590/1170] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:43:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:43:30.717920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91758c33-5935-4f6f-9908-1c49d13b34b4 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:30.718465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91758c33-5935-4f6f-9908-1c49d13b34b4 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:30.718567 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91758c33-5935-4f6f-9908-1c49d13b34b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:30.736266 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-91758c33-5935-4f6f-9908-1c49d13b34b4 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:30.737469 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-91758c33-5935-4f6f-9908-1c49d13b34b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:30.807393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 581/1171] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:43:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:43:30.831356 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e277b0-9628-401e-8697-84707c6af6fa None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:30.935481 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e277b0-9628-401e-8697-84707c6af6fa None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:30.970184 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458e2768-5152-4841-ac43-428d50bf605b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:30.971002 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e2768-5152-4841-ac43-428d50bf605b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:30.971288 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e2768-5152-4841-ac43-428d50bf605b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:30.971567 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e2768-5152-4841-ac43-428d50bf605b None admin] PATH_INFO: `/v3/domains/fb7fc8fb62c14c4084a0eb6952d98092` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:30.983274 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458e2768-5152-4841-ac43-428d50bf605b None admin] RBAC: Authorizing `identity:get_domain(domain_id=fb7fc8fb62c14c4084a0eb6952d98092)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:30.984763 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458e2768-5152-4841-ac43-428d50bf605b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:30.986516 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 591/1172] 10.222.0.78 () {56 vars in 1183 bytes} [Wed May 25 14:43:30 2022] GET /identity/v3/domains/fb7fc8fb62c14c4084a0eb6952d98092 => generated 267 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:30.992425 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91758c33-5935-4f6f-9908-1c49d13b34b4 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:31.083993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:31.091234 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91758c33-5935-4f6f-9908-1c49d13b34b4 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:31.126326 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e2b0db-1ea1-4112-bc64-91345bbcfd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:31.126984 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2b0db-1ea1-4112-bc64-91345bbcfd4b None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:31.127267 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2b0db-1ea1-4112-bc64-91345bbcfd4b None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:31.127543 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2b0db-1ea1-4112-bc64-91345bbcfd4b None admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:31.128252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e2b0db-1ea1-4112-bc64-91345bbcfd4b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:31.129684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e2b0db-1ea1-4112-bc64-91345bbcfd4b None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:31.155028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 582/1173] 10.222.0.78 () {60 vars in 1141 bytes} [Wed May 25 14:43:30 2022] POST /identity/v3/projects => generated 361 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:43:33.428348 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fff1a-7965-4e2c-834e-2c809883f6e8 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:33.428916 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fff1a-7965-4e2c-834e-2c809883f6e8 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:33.428916 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fff1a-7965-4e2c-834e-2c809883f6e8 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:33.430328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 592/1174] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:43:33.435671 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8202ca3-f74e-44f4-a970-87528742c56d None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:33.436084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8202ca3-f74e-44f4-a970-87528742c56d None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:33.436199 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8202ca3-f74e-44f4-a970-87528742c56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:33.453332 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-f8202ca3-f74e-44f4-a970-87528742c56d None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:33.454532 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f8202ca3-f74e-44f4-a970-87528742c56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:33.521920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 583/1175] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:43:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:43:33.548963 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4fff1a-7965-4e2c-834e-2c809883f6e8 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:33.578824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:33.653933 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4fff1a-7965-4e2c-834e-2c809883f6e8 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:33.688466 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22167dd4-18e1-482f-b8c4-2bd131d9a7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:33.689236 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22167dd4-18e1-482f-b8c4-2bd131d9a7ef None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:33.689519 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22167dd4-18e1-482f-b8c4-2bd131d9a7ef None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:33.689793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22167dd4-18e1-482f-b8c4-2bd131d9a7ef None admin] PATH_INFO: `/v3/domains/fb7fc8fb62c14c4084a0eb6952d98092` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:33.691003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22167dd4-18e1-482f-b8c4-2bd131d9a7ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=fb7fc8fb62c14c4084a0eb6952d98092)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:33.692445 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22167dd4-18e1-482f-b8c4-2bd131d9a7ef None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:33.694107 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 593/1176] 10.222.0.78 () {56 vars in 1183 bytes} [Wed May 25 14:43:33 2022] GET /identity/v3/domains/fb7fc8fb62c14c4084a0eb6952d98092 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:33.700328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8202ca3-f74e-44f4-a970-87528742c56d None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:33.780102 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:33.802453 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8202ca3-f74e-44f4-a970-87528742c56d None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:33.835210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fdcfe-d9d2-4e0a-8f70-f23546e5f76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:33.835870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fdcfe-d9d2-4e0a-8f70-f23546e5f76f None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:33.836739 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fdcfe-d9d2-4e0a-8f70-f23546e5f76f None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:33.837012 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fdcfe-d9d2-4e0a-8f70-f23546e5f76f None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:33.837712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41fdcfe-d9d2-4e0a-8f70-f23546e5f76f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:33.839182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41fdcfe-d9d2-4e0a-8f70-f23546e5f76f None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:33.858265 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 584/1177] 10.222.0.78 () {60 vars in 1132 bytes} [Wed May 25 14:43:33 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:43:36.217540 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751dc471-1edf-4246-be62-76eaa63857ec None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:36.218099 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751dc471-1edf-4246-be62-76eaa63857ec None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:36.218099 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751dc471-1edf-4246-be62-76eaa63857ec None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:36.219425 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 594/1178] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:43:36.225142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73af651-0837-46ef-be9e-a885fd15f892 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:36.225651 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73af651-0837-46ef-be9e-a885fd15f892 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:36.225772 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73af651-0837-46ef-be9e-a885fd15f892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:36.245571 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-b73af651-0837-46ef-be9e-a885fd15f892 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:36.246816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-b73af651-0837-46ef-be9e-a885fd15f892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:36.323746 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 585/1179] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:43:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:43:36.348150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751dc471-1edf-4246-be62-76eaa63857ec None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:36.456576 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:36.464967 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751dc471-1edf-4246-be62-76eaa63857ec None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:36.500877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474ea69-0e3a-4a2d-91bf-4f85cb0312e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:36.501699 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474ea69-0e3a-4a2d-91bf-4f85cb0312e7 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:36.501988 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474ea69-0e3a-4a2d-91bf-4f85cb0312e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:36.502313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474ea69-0e3a-4a2d-91bf-4f85cb0312e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:36.513507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2474ea69-0e3a-4a2d-91bf-4f85cb0312e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:43:36.521196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-2474ea69-0e3a-4a2d-91bf-4f85cb0312e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:43:36.521196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:43:36.521196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:43:36.521196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:43:36.521196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:43:36.521196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:43:36.521196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:43:36.521196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:43:36.521196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:43:36.521196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:43:36.521196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:43:36.521196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:43:36.521196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:43:36.521196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:43:36.521196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:43:36.521196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:43:36.522227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:43:36.522227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:43:36.522227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:43:36.522227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:43:36.522227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:43:36.522227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:43:36.522227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:43:36.522227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:43:36.522227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:43:36.522227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:43:36.522227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:43:36.522227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:43:36.522227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:43:36.522227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:43:36.522227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:43:36.522227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:43:36.523158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:43:36.523158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:43:36.523158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:43:36.523158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:43:36.523158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:43:36.523158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:43:36.523158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:43:36.523158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:43:36.523158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:43:36.523158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:43:36.523158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:43:36.523158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:43:36.523158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 595/1180] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:43:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:43:36.529250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73af651-0837-46ef-be9e-a885fd15f892 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:36.595556 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:36.636080 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73af651-0837-46ef-be9e-a885fd15f892 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:36.674692 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5061b-368f-4265-bfab-2cf40a19431f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:36.675366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5061b-368f-4265-bfab-2cf40a19431f None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:36.675649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5061b-368f-4265-bfab-2cf40a19431f None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:36.675925 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5061b-368f-4265-bfab-2cf40a19431f None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:36.676684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5061b-368f-4265-bfab-2cf40a19431f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:43:36.676989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5061b-368f-4265-bfab-2cf40a19431f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:36.678402 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5061b-368f-4265-bfab-2cf40a19431f None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:36.688078 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 586/1181] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:43:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:36.694251 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474ea69-0e3a-4a2d-91bf-4f85cb0312e7 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:36.799952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474ea69-0e3a-4a2d-91bf-4f85cb0312e7 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:36.839647 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b0120-7ad1-47d9-962c-eaca9685b673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:36.840527 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b0120-7ad1-47d9-962c-eaca9685b673 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:36.840832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b0120-7ad1-47d9-962c-eaca9685b673 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:36.841092 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b0120-7ad1-47d9-962c-eaca9685b673 None admin] PATH_INFO: `/v3/users/ca36e68f4e394a83be4b661e6d181271` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:36.856208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b0120-7ad1-47d9-962c-eaca9685b673 None admin] RBAC: Authorizing `identity:get_user(user_id=ca36e68f4e394a83be4b661e6d181271)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:36.857685 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b0120-7ad1-47d9-962c-eaca9685b673 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:36.859473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 596/1182] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:43:36 2022] GET /identity/v3/users/ca36e68f4e394a83be4b661e6d181271 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:43:36.865944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5061b-368f-4265-bfab-2cf40a19431f None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:36.980111 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5061b-368f-4265-bfab-2cf40a19431f None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:37.007657 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:37.017825 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a6bad-94f7-4270-af36-bfd934f33a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:37.018529 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a6bad-94f7-4270-af36-bfd934f33a85 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:37.018808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a6bad-94f7-4270-af36-bfd934f33a85 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:37.019083 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a6bad-94f7-4270-af36-bfd934f33a85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:37.091426 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a6bad-94f7-4270-af36-bfd934f33a85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:37.092819 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a6bad-94f7-4270-af36-bfd934f33a85 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:37.199954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 587/1183] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:43:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:43:37.206721 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b0120-7ad1-47d9-962c-eaca9685b673 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:37.224423 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:37.321055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b0120-7ad1-47d9-962c-eaca9685b673 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:37.357274 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f74d1-8ca3-4e3e-81d5-ad2ec3f7d3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:37.358078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f74d1-8ca3-4e3e-81d5-ad2ec3f7d3ef None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:37.358365 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f74d1-8ca3-4e3e-81d5-ad2ec3f7d3ef None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:37.358642 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f74d1-8ca3-4e3e-81d5-ad2ec3f7d3ef None admin] PATH_INFO: `/v3/projects/27ff1391ab9c48d3a82013581bd48e3e` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:37.359851 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f74d1-8ca3-4e3e-81d5-ad2ec3f7d3ef None admin] RBAC: Authorizing `identity:get_project(project_id=27ff1391ab9c48d3a82013581bd48e3e)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:37.361310 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f74d1-8ca3-4e3e-81d5-ad2ec3f7d3ef None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:37.363094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 597/1184] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:43:37 2022] GET /identity/v3/projects/27ff1391ab9c48d3a82013581bd48e3e => generated 361 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:37.372801 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a6bad-94f7-4270-af36-bfd934f33a85 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:37.400587 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:37.481622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a6bad-94f7-4270-af36-bfd934f33a85 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:37.518559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78d99ad-57ab-4fb0-93ab-00d120f76707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:37.519217 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d99ad-57ab-4fb0-93ab-00d120f76707 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:37.519498 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d99ad-57ab-4fb0-93ab-00d120f76707 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:37.519771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d99ad-57ab-4fb0-93ab-00d120f76707 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:37.520604 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78d99ad-57ab-4fb0-93ab-00d120f76707 None admin] RBAC: Adding query filter params (role.id=e566b37cd92f467198b5dc40f183effc, scope.project.id=27ff1391ab9c48d3a82013581bd48e3e, user.id=ca36e68f4e394a83be4b661e6d181271) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:43:37.520911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78d99ad-57ab-4fb0-93ab-00d120f76707 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:37.522374 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78d99ad-57ab-4fb0-93ab-00d120f76707 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:37.528222 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 588/1185] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:43:37 2022] GET /identity/v3/role_assignments?user.id=ca36e68f4e394a83be4b661e6d181271&scope.project.id=27ff1391ab9c48d3a82013581bd48e3e&role.id=e566b37cd92f467198b5dc40f183effc => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:43:39.785290 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768d928-74ba-493e-a9c8-21de4017a30e None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:39.785848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768d928-74ba-493e-a9c8-21de4017a30e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:39.785848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768d928-74ba-493e-a9c8-21de4017a30e None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:39.787240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 598/1186] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:43:39.793609 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a1cbc-f6e9-4a80-8d64-15315ce53b82 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:39.794167 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a1cbc-f6e9-4a80-8d64-15315ce53b82 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:39.794260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a1cbc-f6e9-4a80-8d64-15315ce53b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:39.812665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-ce3a1cbc-f6e9-4a80-8d64-15315ce53b82 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:39.814066 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-ce3a1cbc-f6e9-4a80-8d64-15315ce53b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:39.886791 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 589/1187] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:43:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:43:39.911661 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768d928-74ba-493e-a9c8-21de4017a30e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:40.002235 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:40.022515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768d928-74ba-493e-a9c8-21de4017a30e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:40.060209 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb8962-d380-4969-a729-ab42e3663db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:40.060919 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb8962-d380-4969-a729-ab42e3663db0 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:40.061203 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb8962-d380-4969-a729-ab42e3663db0 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:40.061474 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb8962-d380-4969-a729-ab42e3663db0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:40.072828 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb8962-d380-4969-a729-ab42e3663db0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:43:40.080456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-c9fb8962-d380-4969-a729-ab42e3663db0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:43:40.080456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:43:40.080456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:43:40.080456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:43:40.080456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:43:40.080456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:43:40.080456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:43:40.080456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:43:40.080456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:43:40.080456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:43:40.080456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:43:40.080456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:43:40.080456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:43:40.080456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:43:40.080456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:43:40.080456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:43:40.081601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:43:40.081601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:43:40.081601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:43:40.081601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:43:40.081601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:43:40.081601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:43:40.081601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:43:40.081601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:43:40.081601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:43:40.081601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:43:40.081601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:43:40.081601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:43:40.081601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:43:40.081601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:43:40.081601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:43:40.081601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:43:40.082756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:43:40.082756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:43:40.082756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:43:40.082756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:43:40.082756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:43:40.082756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:43:40.082756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:43:40.082756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:43:40.082756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:43:40.082756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:43:40.082756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:43:40.082756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:43:40.082756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 599/1188] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:43:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:43:40.087948 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a1cbc-f6e9-4a80-8d64-15315ce53b82 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:40.108984 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:40.191263 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a1cbc-f6e9-4a80-8d64-15315ce53b82 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:40.224886 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffbec73-e71b-46d8-bb0c-d63807a6db83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:40.225546 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbec73-e71b-46d8-bb0c-d63807a6db83 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:40.225829 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbec73-e71b-46d8-bb0c-d63807a6db83 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:40.226661 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbec73-e71b-46d8-bb0c-d63807a6db83 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:40.227410 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffbec73-e71b-46d8-bb0c-d63807a6db83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:43:40.227715 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffbec73-e71b-46d8-bb0c-d63807a6db83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:40.229079 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffbec73-e71b-46d8-bb0c-d63807a6db83 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:40.237217 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 590/1189] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:43:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:40.243820 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb8962-d380-4969-a729-ab42e3663db0 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:40.346049 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb8962-d380-4969-a729-ab42e3663db0 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:40.379605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823395f0-31df-49b0-8b66-8a8afa621cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:40.380467 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823395f0-31df-49b0-8b66-8a8afa621cb4 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:40.380753 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823395f0-31df-49b0-8b66-8a8afa621cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:40.381032 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823395f0-31df-49b0-8b66-8a8afa621cb4 None admin] PATH_INFO: `/v3/users/ca36e68f4e394a83be4b661e6d181271` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:40.382240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823395f0-31df-49b0-8b66-8a8afa621cb4 None admin] RBAC: Authorizing `identity:get_user(user_id=ca36e68f4e394a83be4b661e6d181271)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:40.383746 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823395f0-31df-49b0-8b66-8a8afa621cb4 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:40.385406 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 600/1190] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:43:40 2022] GET /identity/v3/users/ca36e68f4e394a83be4b661e6d181271 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:40.391241 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffbec73-e71b-46d8-bb0c-d63807a6db83 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:40.475544 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:40.496734 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffbec73-e71b-46d8-bb0c-d63807a6db83 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:40.533845 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551a6b1-c7d2-4c7f-8676-d3f2242beec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:40.534748 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551a6b1-c7d2-4c7f-8676-d3f2242beec4 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:40.535027 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551a6b1-c7d2-4c7f-8676-d3f2242beec4 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:40.535299 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551a6b1-c7d2-4c7f-8676-d3f2242beec4 None admin] PATH_INFO: `/v3/projects/27ff1391ab9c48d3a82013581bd48e3e` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:40.536543 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a551a6b1-c7d2-4c7f-8676-d3f2242beec4 None admin] RBAC: Authorizing `identity:get_project(project_id=27ff1391ab9c48d3a82013581bd48e3e)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:40.538044 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a551a6b1-c7d2-4c7f-8676-d3f2242beec4 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:40.540341 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 591/1191] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:43:40 2022] GET /identity/v3/projects/27ff1391ab9c48d3a82013581bd48e3e => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:40.546957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823395f0-31df-49b0-8b66-8a8afa621cb4 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:40.561051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:40.653696 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823395f0-31df-49b0-8b66-8a8afa621cb4 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:40.688038 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc8183-4a34-4f22-9604-114b0f6fd695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:40.688930 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc8183-4a34-4f22-9604-114b0f6fd695 None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:40.689219 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc8183-4a34-4f22-9604-114b0f6fd695 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:40.689501 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc8183-4a34-4f22-9604-114b0f6fd695 None admin] PATH_INFO: `/v3/projects/27ff1391ab9c48d3a82013581bd48e3e/users/ca36e68f4e394a83be4b661e6d181271/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:40.691530 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc8183-4a34-4f22-9604-114b0f6fd695 None admin] RBAC: Authorizing `identity:create_grant(project_id=27ff1391ab9c48d3a82013581bd48e3e, user_id=ca36e68f4e394a83be4b661e6d181271, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:40.692993 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc8183-4a34-4f22-9604-114b0f6fd695 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:40.708133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 601/1192] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:43:40 2022] PUT /identity/v3/projects/27ff1391ab9c48d3a82013581bd48e3e/users/ca36e68f4e394a83be4b661e6d181271/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:43:42.951442 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c86f9-5c97-4944-8e21-66bb3e83b838 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:42.951967 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c86f9-5c97-4944-8e21-66bb3e83b838 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:42.952084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c86f9-5c97-4944-8e21-66bb3e83b838 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:42.953377 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 592/1193] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:43:42.959061 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18a5ec-4a63-4c5e-97db-2feb521ee0d6 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:42.959562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18a5ec-4a63-4c5e-97db-2feb521ee0d6 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:42.959655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18a5ec-4a63-4c5e-97db-2feb521ee0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:42.978438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-ad18a5ec-4a63-4c5e-97db-2feb521ee0d6 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:42.979225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ad18a5ec-4a63-4c5e-97db-2feb521ee0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:43.042776 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:43.048234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 602/1194] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:43:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:43:43.071368 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c86f9-5c97-4944-8e21-66bb3e83b838 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:43.172166 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c86f9-5c97-4944-8e21-66bb3e83b838 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:43.204185 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540ce0c-e70e-42c3-8a7a-2192d219880c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:43.204969 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540ce0c-e70e-42c3-8a7a-2192d219880c None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:43.205251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540ce0c-e70e-42c3-8a7a-2192d219880c None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:43.205525 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540ce0c-e70e-42c3-8a7a-2192d219880c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:43.216342 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540ce0c-e70e-42c3-8a7a-2192d219880c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:43:43.223923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-6540ce0c-e70e-42c3-8a7a-2192d219880c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:43:43.223923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:43:43.223923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:43:43.223923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:43:43.223923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:43:43.223923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:43:43.223923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:43:43.223923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:43:43.223923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:43:43.223923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:43:43.223923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:43:43.223923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:43:43.223923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:43:43.223923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:43:43.223923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:43:43.223923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:43:43.225008 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:43:43.225008 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:43:43.225008 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:43:43.225008 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:43:43.225008 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:43:43.225008 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:43:43.225008 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:43:43.225008 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:43:43.225008 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:43:43.225008 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:43:43.225008 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:43:43.225008 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:43:43.225008 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:43:43.225008 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:43:43.225008 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:43:43.225008 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:43:43.226712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:43:43.226712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:43:43.226712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:43:43.226712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:43:43.226712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:43:43.226712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:43:43.226712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:43:43.226712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:43:43.226712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:43:43.226712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:43:43.226712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:43:43.226712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:43:43.226712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 593/1195] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:43:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:43:43.231749 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad18a5ec-4a63-4c5e-97db-2feb521ee0d6 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:43.330111 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad18a5ec-4a63-4c5e-97db-2feb521ee0d6 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:43.364399 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bbcc7-3df3-446c-bf13-200c286f0d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:43.365064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bbcc7-3df3-446c-bf13-200c286f0d7c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:43.365352 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bbcc7-3df3-446c-bf13-200c286f0d7c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:43.365631 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bbcc7-3df3-446c-bf13-200c286f0d7c None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:43.366403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bbcc7-3df3-446c-bf13-200c286f0d7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:43:43.366713 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bbcc7-3df3-446c-bf13-200c286f0d7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:43.368098 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bbcc7-3df3-446c-bf13-200c286f0d7c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:43.376230 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 603/1196] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:43:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:43.382061 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540ce0c-e70e-42c3-8a7a-2192d219880c None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:43.394744 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:43.481741 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540ce0c-e70e-42c3-8a7a-2192d219880c None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:43.515137 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb14170-9675-479f-9136-ca39d449ac0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:43.515998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb14170-9675-479f-9136-ca39d449ac0d None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:43.516278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb14170-9675-479f-9136-ca39d449ac0d None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:43.516552 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb14170-9675-479f-9136-ca39d449ac0d None admin] PATH_INFO: `/v3/users/ca36e68f4e394a83be4b661e6d181271` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:43.517722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb14170-9675-479f-9136-ca39d449ac0d None admin] RBAC: Authorizing `identity:get_user(user_id=ca36e68f4e394a83be4b661e6d181271)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:43.519182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb14170-9675-479f-9136-ca39d449ac0d None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:43.520797 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 594/1197] 10.222.0.78 () {56 vars in 1177 bytes} [Wed May 25 14:43:43 2022] GET /identity/v3/users/ca36e68f4e394a83be4b661e6d181271 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:43.527177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bbcc7-3df3-446c-bf13-200c286f0d7c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:43.587108 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:43.625924 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bbcc7-3df3-446c-bf13-200c286f0d7c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:43.660969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389e629-fcff-4c39-a7c4-fedb565fc3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:43.661618 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389e629-fcff-4c39-a7c4-fedb565fc3c0 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:43.661902 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389e629-fcff-4c39-a7c4-fedb565fc3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:43.662210 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389e629-fcff-4c39-a7c4-fedb565fc3c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:43.732922 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4389e629-fcff-4c39-a7c4-fedb565fc3c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:43.734336 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4389e629-fcff-4c39-a7c4-fedb565fc3c0 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:43.795681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:43.835484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 604/1198] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:43:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:43:43.842494 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb14170-9675-479f-9136-ca39d449ac0d None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:43.922305 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:43.945863 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb14170-9675-479f-9136-ca39d449ac0d None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:43.979540 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c56a64-9c7f-42ab-9df0-1fb49f396e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:43.980405 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c56a64-9c7f-42ab-9df0-1fb49f396e3a None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:43.980684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c56a64-9c7f-42ab-9df0-1fb49f396e3a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:43.980958 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c56a64-9c7f-42ab-9df0-1fb49f396e3a None admin] PATH_INFO: `/v3/projects/27ff1391ab9c48d3a82013581bd48e3e` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:43.982172 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c56a64-9c7f-42ab-9df0-1fb49f396e3a None admin] RBAC: Authorizing `identity:get_project(project_id=27ff1391ab9c48d3a82013581bd48e3e)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:43.983654 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c56a64-9c7f-42ab-9df0-1fb49f396e3a None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:43.985305 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 595/1199] 10.222.0.78 () {56 vars in 1186 bytes} [Wed May 25 14:43:43 2022] GET /identity/v3/projects/27ff1391ab9c48d3a82013581bd48e3e => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:43.992248 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389e629-fcff-4c39-a7c4-fedb565fc3c0 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:44.089170 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389e629-fcff-4c39-a7c4-fedb565fc3c0 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:44.124610 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12efd4c1-cb7f-4310-b39d-291cbce5de3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:44.125200 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efd4c1-cb7f-4310-b39d-291cbce5de3a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:44.125542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efd4c1-cb7f-4310-b39d-291cbce5de3a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:44.125766 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efd4c1-cb7f-4310-b39d-291cbce5de3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:44.126619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12efd4c1-cb7f-4310-b39d-291cbce5de3a None admin] RBAC: Adding query filter params (role.id=e566b37cd92f467198b5dc40f183effc, scope.project.id=27ff1391ab9c48d3a82013581bd48e3e, user.id=ca36e68f4e394a83be4b661e6d181271) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:43:44.126930 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12efd4c1-cb7f-4310-b39d-291cbce5de3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:44.128385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12efd4c1-cb7f-4310-b39d-291cbce5de3a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:44.134478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 605/1200] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:43:43 2022] GET /identity/v3/role_assignments?user.id=ca36e68f4e394a83be4b661e6d181271&scope.project.id=27ff1391ab9c48d3a82013581bd48e3e&role.id=e566b37cd92f467198b5dc40f183effc => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:53.974453 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=27 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:54.132332 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=87 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:46.480127 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340bc2c-5c87-491c-b9ad-8eec5cc64ffe None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:46.480752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340bc2c-5c87-491c-b9ad-8eec5cc64ffe None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:46.480752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340bc2c-5c87-491c-b9ad-8eec5cc64ffe None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:46.483030 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 596/1201] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:44:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:44:46.490944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fd9b3-80a9-4083-91cf-0f2ff642a220 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:46.494429 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fd9b3-80a9-4083-91cf-0f2ff642a220 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:46.494429 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fd9b3-80a9-4083-91cf-0f2ff642a220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:46.514230 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-075fd9b3-80a9-4083-91cf-0f2ff642a220 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:44:46.515819 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-075fd9b3-80a9-4083-91cf-0f2ff642a220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:46.634746 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 606/1202] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:44:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:44:46.661219 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340bc2c-5c87-491c-b9ad-8eec5cc64ffe None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:46.797586 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340bc2c-5c87-491c-b9ad-8eec5cc64ffe None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:46.844462 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15e08d-c265-40b8-9760-2213a8b899d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:46.845283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15e08d-c265-40b8-9760-2213a8b899d1 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:46.845562 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15e08d-c265-40b8-9760-2213a8b899d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:46.845832 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15e08d-c265-40b8-9760-2213a8b899d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:46.863412 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15e08d-c265-40b8-9760-2213a8b899d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:46.864886 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15e08d-c265-40b8-9760-2213a8b899d1 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:46.886002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-bf15e08d-c265-40b8-9760-2213a8b899d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:44:46.886002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:44:46.886002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:44:46.886002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:44:46.886002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:44:46.886002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:44:46.886002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:44:46.886002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:44:46.886002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:44:46.886002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:44:46.886002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:44:46.886002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:44:46.886002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:44:46.886002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:44:46.886002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:44:46.887099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:44:46.887099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:44:46.887099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:44:46.887099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:44:46.887099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:44:46.887099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:44:46.887099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:44:46.887099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:44:46.887099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:44:46.887099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:44:46.887099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:44:46.887099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:44:46.887099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:44:46.887099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:44:46.887099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:44:46.887099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:44:46.887099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:44:46.888068 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:44:46.888068 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:44:46.888068 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:44:46.888068 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:44:46.888068 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 597/1203] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:44:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:44:46.895977 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075fd9b3-80a9-4083-91cf-0f2ff642a220 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:47.039935 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075fd9b3-80a9-4083-91cf-0f2ff642a220 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:47.084841 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:47.093892 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059896a-9f32-4070-babd-0a2a71d0ccf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:47.094724 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059896a-9f32-4070-babd-0a2a71d0ccf1 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:47.095162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059896a-9f32-4070-babd-0a2a71d0ccf1 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:47.095610 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059896a-9f32-4070-babd-0a2a71d0ccf1 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:47.096540 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059896a-9f32-4070-babd-0a2a71d0ccf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:44:47.096813 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059896a-9f32-4070-babd-0a2a71d0ccf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:47.098475 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059896a-9f32-4070-babd-0a2a71d0ccf1 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:47.115114 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 607/1204] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:44:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:44:47.121291 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15e08d-c265-40b8-9760-2213a8b899d1 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:47.204870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:47.262198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15e08d-c265-40b8-9760-2213a8b899d1 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:47.320302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9171a63-b258-48b9-b280-457e9113bb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:47.321045 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9171a63-b258-48b9-b280-457e9113bb49 None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:47.321379 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9171a63-b258-48b9-b280-457e9113bb49 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:47.321671 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9171a63-b258-48b9-b280-457e9113bb49 None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:47.322443 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9171a63-b258-48b9-b280-457e9113bb49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:47.324032 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9171a63-b258-48b9-b280-457e9113bb49 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:47.355100 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 598/1205] 10.222.0.78 () {60 vars in 1132 bytes} [Wed May 25 14:44:47 2022] POST /identity/v3/users => generated 251 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:44:49.936856 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156700e0-86ba-4a0c-b87c-47590138b12d None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:49.937415 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156700e0-86ba-4a0c-b87c-47590138b12d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:49.937415 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156700e0-86ba-4a0c-b87c-47590138b12d None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:49.938859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 608/1206] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:44:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:44:49.946438 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5187265-20cd-4454-8f7c-08a76a52748a None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:49.946993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5187265-20cd-4454-8f7c-08a76a52748a None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:49.946993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5187265-20cd-4454-8f7c-08a76a52748a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:49.970860 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-d5187265-20cd-4454-8f7c-08a76a52748a None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:44:49.972774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d5187265-20cd-4454-8f7c-08a76a52748a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:50.067157 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 599/1207] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:44:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:44:50.091295 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156700e0-86ba-4a0c-b87c-47590138b12d None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:50.209014 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156700e0-86ba-4a0c-b87c-47590138b12d None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:50.248518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d28d2-2da7-4bcf-ae31-f832e13ae8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:50.249428 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d28d2-2da7-4bcf-ae31-f832e13ae8d8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:50.249711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d28d2-2da7-4bcf-ae31-f832e13ae8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:50.249994 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d28d2-2da7-4bcf-ae31-f832e13ae8d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:50.263928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d28d2-2da7-4bcf-ae31-f832e13ae8d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:44:50.272351 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-599d28d2-2da7-4bcf-ae31-f832e13ae8d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:44:50.272351 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:44:50.272351 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:44:50.272351 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:44:50.272351 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:44:50.272351 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:44:50.272351 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:44:50.272351 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:44:50.272351 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:44:50.272351 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:44:50.272351 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:44:50.272351 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:44:50.272351 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:44:50.272351 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:44:50.272351 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:44:50.272351 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:44:50.273710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:44:50.273710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:44:50.273710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:44:50.273710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:44:50.273710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:44:50.273710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:44:50.273710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:44:50.273710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:44:50.273710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:44:50.273710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:44:50.273710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:44:50.273710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:44:50.273710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:44:50.273710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:44:50.273710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:44:50.273710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:44:50.275031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:44:50.275031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:44:50.275031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:44:50.275031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:44:50.275031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:44:50.275031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:44:50.275031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:44:50.275031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:44:50.275031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:44:50.275031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:44:50.275031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:44:50.275031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:44:50.278978 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 609/1208] 10.222.0.78 () {56 vars in 1102 bytes} [Wed May 25 14:44:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:44:50.279980 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5187265-20cd-4454-8f7c-08a76a52748a None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:50.333711 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:50.409816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5187265-20cd-4454-8f7c-08a76a52748a None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:50.449327 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d38663-ad03-4d36-ad15-8cdbaee1aecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:50.451975 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d38663-ad03-4d36-ad15-8cdbaee1aecc None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:50.451975 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d38663-ad03-4d36-ad15-8cdbaee1aecc None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:50.451975 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d38663-ad03-4d36-ad15-8cdbaee1aecc None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:50.451975 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d38663-ad03-4d36-ad15-8cdbaee1aecc None admin] RBAC: Adding query filter params (name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:44:50.451975 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d38663-ad03-4d36-ad15-8cdbaee1aecc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:50.452981 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d38663-ad03-4d36-ad15-8cdbaee1aecc None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:50.462411 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 600/1209] 10.222.0.78 () {56 vars in 1103 bytes} [Wed May 25 14:44:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:44:50.468433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d28d2-2da7-4bcf-ae31-f832e13ae8d8 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:50.533011 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:50.602641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d28d2-2da7-4bcf-ae31-f832e13ae8d8 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:50.644960 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d50795-bd4c-47b8-892f-fc8f529f9b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:50.645798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d50795-bd4c-47b8-892f-fc8f529f9b4a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:50.646263 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d50795-bd4c-47b8-892f-fc8f529f9b4a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:50.646564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d50795-bd4c-47b8-892f-fc8f529f9b4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:50.660228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d50795-bd4c-47b8-892f-fc8f529f9b4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:50.661645 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d50795-bd4c-47b8-892f-fc8f529f9b4a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:50.679327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-f3d50795-bd4c-47b8-892f-fc8f529f9b4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:44:50.679327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:44:50.679327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:44:50.679327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:44:50.679327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:44:50.679327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:44:50.679327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:44:50.679327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:44:50.679327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:44:50.679327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:44:50.679327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:44:50.679327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:44:50.679327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:44:50.679327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:44:50.679327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:44:50.680330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:44:50.680330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:44:50.680330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:44:50.680330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:44:50.680330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:44:50.680330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:44:50.680330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:44:50.680330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:44:50.680330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:44:50.680330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:44:50.680330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:44:50.680330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:44:50.680330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:44:50.680330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:44:50.680330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:44:50.680330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:44:50.680330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:44:50.681233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:44:50.681233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:44:50.681233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:44:50.681233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:44:50.681233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 610/1210] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:44:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:44:50.686311 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d38663-ad03-4d36-ad15-8cdbaee1aecc None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:50.807067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:50.810254 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d38663-ad03-4d36-ad15-8cdbaee1aecc None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:50.855145 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ed6fe-92df-4012-a723-eb7d95463922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:50.855822 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ed6fe-92df-4012-a723-eb7d95463922 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:50.856104 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ed6fe-92df-4012-a723-eb7d95463922 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:50.856430 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ed6fe-92df-4012-a723-eb7d95463922 None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:50.857326 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ed6fe-92df-4012-a723-eb7d95463922 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:44:50.857628 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ed6fe-92df-4012-a723-eb7d95463922 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:50.860206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ed6fe-92df-4012-a723-eb7d95463922 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:50.881744 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 601/1211] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:44:50.883124 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d50795-bd4c-47b8-892f-fc8f529f9b4a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:51.019964 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d50795-bd4c-47b8-892f-fc8f529f9b4a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:51.021940 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:51.068133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ab59d-c6af-46fa-ae55-fb2fc2dc5093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:51.068133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ab59d-c6af-46fa-ae55-fb2fc2dc5093 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:51.068133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ab59d-c6af-46fa-ae55-fb2fc2dc5093 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:51.068133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ab59d-c6af-46fa-ae55-fb2fc2dc5093 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:51.076787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ab59d-c6af-46fa-ae55-fb2fc2dc5093 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:51.078310 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ab59d-c6af-46fa-ae55-fb2fc2dc5093 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:51.095327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-5d9ab59d-c6af-46fa-ae55-fb2fc2dc5093 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 14:44:51.095327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:44:51.095327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:44:51.095327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:44:51.095327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:44:51.095327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:44:51.095327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:44:51.095327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:44:51.095327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:44:51.095327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:44:51.095327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:44:51.095327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:44:51.095327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:44:51.095327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:44:51.095327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:44:51.095327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:44:51.096486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:44:51.096486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:44:51.096486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:44:51.096486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:44:51.096486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:44:51.096486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:44:51.096486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:44:51.096486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:44:51.096486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:44:51.096486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:44:51.096486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:44:51.096486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:44:51.096486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:44:51.096486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:44:51.096486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:44:51.096486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:44:51.096486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:44:51.096486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:44:51.097480 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:44:51.097480 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:44:51.097480 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:44:51.097480 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:44:51.097480 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:44:51.097480 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:44:51.097480 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:44:51.097480 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:44:51.097480 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 14:44:51.097480 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:44:51.097480 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 611/1212] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:44:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:44:51.102881 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ed6fe-92df-4012-a723-eb7d95463922 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:51.245952 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ed6fe-92df-4012-a723-eb7d95463922 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:51.294949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a393f-d332-47ec-b9e4-fb070addf35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:51.295891 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a393f-d332-47ec-b9e4-fb070addf35e None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:51.296266 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a393f-d332-47ec-b9e4-fb070addf35e None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:51.296657 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a393f-d332-47ec-b9e4-fb070addf35e None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:51.297663 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a393f-d332-47ec-b9e4-fb070addf35e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:44:51.297987 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a393f-d332-47ec-b9e4-fb070addf35e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:51.300112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a393f-d332-47ec-b9e4-fb070addf35e None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:51.328262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 602/1213] 10.222.0.78 () {56 vars in 1143 bytes} [Wed May 25 14:44:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:44:51.337245 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ab59d-c6af-46fa-ae55-fb2fc2dc5093 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:51.489909 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ab59d-c6af-46fa-ae55-fb2fc2dc5093 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:51.539683 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4122e2f5-8687-4e9e-9f99-e464c95a15af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:51.540577 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4122e2f5-8687-4e9e-9f99-e464c95a15af None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:51.541062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4122e2f5-8687-4e9e-9f99-e464c95a15af None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:51.541592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4122e2f5-8687-4e9e-9f99-e464c95a15af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:51.587794 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:51.642455 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4122e2f5-8687-4e9e-9f99-e464c95a15af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:51.645232 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4122e2f5-8687-4e9e-9f99-e464c95a15af None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:51.799549 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 612/1214] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:44:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:44:51.801073 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a393f-d332-47ec-b9e4-fb070addf35e None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:51.824678 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:51.936669 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a393f-d332-47ec-b9e4-fb070addf35e None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:51.985064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41954a19-9bf5-4c74-919b-b3b4ddeb4105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:51.986371 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41954a19-9bf5-4c74-919b-b3b4ddeb4105 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:51.987087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41954a19-9bf5-4c74-919b-b3b4ddeb4105 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:51.988045 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41954a19-9bf5-4c74-919b-b3b4ddeb4105 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:52.013802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41954a19-9bf5-4c74-919b-b3b4ddeb4105 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:52.016553 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41954a19-9bf5-4c74-919b-b3b4ddeb4105 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:52.053431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-41954a19-9bf5-4c74-919b-b3b4ddeb4105 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:44:52.053431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:44:52.053431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:44:52.053431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:44:52.053431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:44:52.053431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:44:52.053431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:44:52.053431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:44:52.053431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:44:52.053431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:44:52.053431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:44:52.053431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:44:52.053431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:44:52.053431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:44:52.053431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:44:52.055142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:44:52.055142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:44:52.055142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:44:52.055142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:44:52.055142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:44:52.055142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:44:52.055142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:44:52.055142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:44:52.055142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:44:52.055142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:44:52.055142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:44:52.055142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:44:52.055142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:44:52.055142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:44:52.055142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:44:52.055142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:44:52.055142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:44:52.056021 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:44:52.056021 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:44:52.056021 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:44:52.056021 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:44:52.056021 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 603/1215] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:44:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:44:52.061201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4122e2f5-8687-4e9e-9f99-e464c95a15af None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:52.141821 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:52.192846 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4122e2f5-8687-4e9e-9f99-e464c95a15af None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:52.245036 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de0fd8-39f6-46ef-bfd6-4be0246c92b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:52.246098 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de0fd8-39f6-46ef-bfd6-4be0246c92b7 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:52.246613 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de0fd8-39f6-46ef-bfd6-4be0246c92b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:52.247124 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de0fd8-39f6-46ef-bfd6-4be0246c92b7 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:52.248495 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de0fd8-39f6-46ef-bfd6-4be0246c92b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:44:52.248495 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de0fd8-39f6-46ef-bfd6-4be0246c92b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:52.250289 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de0fd8-39f6-46ef-bfd6-4be0246c92b7 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:52.269987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 613/1216] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:44:52.271693 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41954a19-9bf5-4c74-919b-b3b4ddeb4105 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:52.381910 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:52.426133 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41954a19-9bf5-4c74-919b-b3b4ddeb4105 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:52.470092 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c3b69-2dec-49b1-b837-30131f99e4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:52.470830 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c3b69-2dec-49b1-b837-30131f99e4b5 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:52.471111 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c3b69-2dec-49b1-b837-30131f99e4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:52.471415 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c3b69-2dec-49b1-b837-30131f99e4b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:52.479690 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c3b69-2dec-49b1-b837-30131f99e4b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:52.481422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c3b69-2dec-49b1-b837-30131f99e4b5 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:52.496971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-271c3b69-2dec-49b1-b837-30131f99e4b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:44:52.496971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:44:52.496971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:44:52.496971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:44:52.496971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:44:52.496971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:44:52.496971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:44:52.496971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:44:52.496971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:44:52.496971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:44:52.496971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:44:52.496971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:44:52.496971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:44:52.496971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:44:52.496971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:44:52.497966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:44:52.497966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:44:52.497966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:44:52.497966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:44:52.497966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:44:52.497966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:44:52.497966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:44:52.497966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:44:52.497966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:44:52.497966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:44:52.497966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:44:52.497966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:44:52.497966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:44:52.497966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:44:52.497966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:44:52.497966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:44:52.497966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:44:52.499558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:44:52.499558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:44:52.499558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:44:52.499558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:44:52.499558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:44:52.499558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:44:52.499558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:44:52.499558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:44:52.499558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 604/1217] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:44:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:44:52.504976 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de0fd8-39f6-46ef-bfd6-4be0246c92b7 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:52.634374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de0fd8-39f6-46ef-bfd6-4be0246c92b7 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:52.664696 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:52.682109 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07edf2c-89e5-493d-8c03-d62d6798325d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:52.682995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07edf2c-89e5-493d-8c03-d62d6798325d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:52.683377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07edf2c-89e5-493d-8c03-d62d6798325d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:52.683693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07edf2c-89e5-493d-8c03-d62d6798325d None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:52.684613 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07edf2c-89e5-493d-8c03-d62d6798325d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:44:52.684939 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07edf2c-89e5-493d-8c03-d62d6798325d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:52.686603 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07edf2c-89e5-493d-8c03-d62d6798325d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:52.701918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 614/1218] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:44:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:44:52.707865 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c3b69-2dec-49b1-b837-30131f99e4b5 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:52.821314 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c3b69-2dec-49b1-b837-30131f99e4b5 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:52.852298 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:52.862060 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08816929-d6d0-4fe4-b78e-555faf7c1a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:52.862060 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08816929-d6d0-4fe4-b78e-555faf7c1a59 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:52.862060 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08816929-d6d0-4fe4-b78e-555faf7c1a59 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:52.862060 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08816929-d6d0-4fe4-b78e-555faf7c1a59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:52.863060 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08816929-d6d0-4fe4-b78e-555faf7c1a59 None admin] RBAC: Adding query filter params (role.id=336f27e7f8624a30b30cef9acada97a0, scope.project.id=908a13d2b49544fbaf760b07e213bb7d, user.id=63789c0a113447ff90933d447cf5bac3) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:44:52.863156 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08816929-d6d0-4fe4-b78e-555faf7c1a59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:52.864686 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08816929-d6d0-4fe4-b78e-555faf7c1a59 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:52.871808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 605/1219] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:44:52 2022] GET /identity/v3/role_assignments?user.id=63789c0a113447ff90933d447cf5bac3&scope.project.id=908a13d2b49544fbaf760b07e213bb7d&role.id=336f27e7f8624a30b30cef9acada97a0 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:44:55.318816 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92961d0-bca0-4b00-9551-d354278da7ab None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:55.319386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92961d0-bca0-4b00-9551-d354278da7ab None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:55.319493 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92961d0-bca0-4b00-9551-d354278da7ab None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:55.321179 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 615/1220] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:44:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:44:55.327610 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f5501-9b72-47fb-ba23-a1fccaf03b82 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:55.328106 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f5501-9b72-47fb-ba23-a1fccaf03b82 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:55.328218 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f5501-9b72-47fb-ba23-a1fccaf03b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:55.345793 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-8a2f5501-9b72-47fb-ba23-a1fccaf03b82 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:44:55.347492 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-8a2f5501-9b72-47fb-ba23-a1fccaf03b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:55.423679 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 606/1221] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:44:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:44:55.446883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92961d0-bca0-4b00-9551-d354278da7ab None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:55.561776 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92961d0-bca0-4b00-9551-d354278da7ab None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:55.598816 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a097a-d09a-407e-98fe-f1de91be1de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:55.599555 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a097a-d09a-407e-98fe-f1de91be1de4 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:55.599921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a097a-d09a-407e-98fe-f1de91be1de4 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:55.600118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a097a-d09a-407e-98fe-f1de91be1de4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:55.611793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9a097a-d09a-407e-98fe-f1de91be1de4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:44:55.619864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-6f9a097a-d09a-407e-98fe-f1de91be1de4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:44:55.619864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:44:55.619864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:44:55.619864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:44:55.619864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:44:55.619864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:44:55.619864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:44:55.619864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:44:55.619864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:44:55.619864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:44:55.619864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:44:55.619864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:44:55.619864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:44:55.619864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:44:55.619864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:44:55.619864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:44:55.621106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:44:55.621106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:44:55.621106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:44:55.621106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:44:55.621106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:44:55.621106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:44:55.621106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:44:55.621106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:44:55.621106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:44:55.621106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:44:55.621106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:44:55.621106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:44:55.621106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:44:55.621106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:44:55.621106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:44:55.621106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:44:55.627101 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:44:55.627101 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:44:55.627101 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:44:55.627101 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:44:55.627101 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:44:55.627101 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:44:55.627101 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:44:55.627101 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:44:55.627101 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:44:55.627101 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:44:55.627101 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:44:55.627101 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:44:55.627101 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 616/1222] 10.222.0.78 () {56 vars in 1102 bytes} [Wed May 25 14:44:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:44:55.628008 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f5501-9b72-47fb-ba23-a1fccaf03b82 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:55.733015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f5501-9b72-47fb-ba23-a1fccaf03b82 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:55.760144 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:55.772822 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a35eae-f06b-49f3-849e-290dd4e47280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:55.773559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a35eae-f06b-49f3-849e-290dd4e47280 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:55.773840 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a35eae-f06b-49f3-849e-290dd4e47280 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:55.774149 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a35eae-f06b-49f3-849e-290dd4e47280 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:55.775002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a35eae-f06b-49f3-849e-290dd4e47280 None admin] RBAC: Adding query filter params (name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:44:55.775312 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a35eae-f06b-49f3-849e-290dd4e47280 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:55.776689 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a35eae-f06b-49f3-849e-290dd4e47280 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:55.786494 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 607/1223] 10.222.0.78 () {56 vars in 1103 bytes} [Wed May 25 14:44:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:44:55.794630 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a097a-d09a-407e-98fe-f1de91be1de4 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:55.849588 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:55.919328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a097a-d09a-407e-98fe-f1de91be1de4 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:55.957076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13f3b7-270b-46af-bc49-8531e6d42212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:55.957877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13f3b7-270b-46af-bc49-8531e6d42212 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:55.958192 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13f3b7-270b-46af-bc49-8531e6d42212 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:55.958472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13f3b7-270b-46af-bc49-8531e6d42212 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:55.973143 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13f3b7-270b-46af-bc49-8531e6d42212 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:55.974816 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13f3b7-270b-46af-bc49-8531e6d42212 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:55.992433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-2a13f3b7-270b-46af-bc49-8531e6d42212 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:44:55.992433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:44:55.992433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:44:55.992433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:44:55.992433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:44:55.992433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:44:55.992433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:44:55.992433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:44:55.992433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:44:55.992433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:44:55.992433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:44:55.992433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:44:55.992433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:44:55.992433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:44:55.992433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:44:55.993494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:44:55.993494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:44:55.993494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:44:55.993494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:44:55.993494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:44:55.993494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:44:55.993494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:44:55.993494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:44:55.993494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:44:55.993494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:44:55.993494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:44:55.993494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:44:55.993494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:44:55.993494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:44:55.993494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:44:55.993494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:44:55.993494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:44:55.994458 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:44:55.994458 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:44:55.994458 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:44:55.994458 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:44:55.994458 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 617/1224] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:44:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:44:56.001290 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a35eae-f06b-49f3-849e-290dd4e47280 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:56.125358 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a35eae-f06b-49f3-849e-290dd4e47280 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:56.164609 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6e105-3f79-42e9-a490-6a1f786deaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:56.165776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6e105-3f79-42e9-a490-6a1f786deaf1 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:56.166316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6e105-3f79-42e9-a490-6a1f786deaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:56.166794 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6e105-3f79-42e9-a490-6a1f786deaf1 None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:56.169307 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6e105-3f79-42e9-a490-6a1f786deaf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:44:56.169307 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6e105-3f79-42e9-a490-6a1f786deaf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:56.170797 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6e105-3f79-42e9-a490-6a1f786deaf1 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:56.186402 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 608/1225] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:44:56.192752 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13f3b7-270b-46af-bc49-8531e6d42212 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:56.320259 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:56.328757 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13f3b7-270b-46af-bc49-8531e6d42212 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:56.368211 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cbc38-d484-4495-b0d2-d5b93d6c925f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:56.369041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cbc38-d484-4495-b0d2-d5b93d6c925f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:56.369319 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cbc38-d484-4495-b0d2-d5b93d6c925f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:56.369593 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cbc38-d484-4495-b0d2-d5b93d6c925f None admin] PATH_INFO: `/v3/users/placement` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:56.377943 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78cbc38-d484-4495-b0d2-d5b93d6c925f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:56.379366 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78cbc38-d484-4495-b0d2-d5b93d6c925f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:56.396390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-d78cbc38-d484-4495-b0d2-d5b93d6c925f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 14:44:56.396390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:44:56.396390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:44:56.396390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:44:56.396390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:44:56.396390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:44:56.396390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:44:56.396390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:44:56.396390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:44:56.396390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:44:56.396390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:44:56.396390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:44:56.396390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:44:56.396390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:44:56.396390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:44:56.396390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:44:56.397706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:44:56.397706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:44:56.397706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:44:56.397706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:44:56.397706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:44:56.397706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:44:56.397706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:44:56.397706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:44:56.397706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:44:56.397706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:44:56.397706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:44:56.397706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:44:56.397706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:44:56.397706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:44:56.397706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:44:56.397706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:44:56.397706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:44:56.397706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:44:56.398984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:44:56.398984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:44:56.398984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:44:56.398984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:44:56.398984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:44:56.398984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:44:56.398984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:44:56.398984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:44:56.398984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 14:44:56.398984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:44:56.398984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 618/1226] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:44:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:44:56.404257 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6e105-3f79-42e9-a490-6a1f786deaf1 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:56.456907 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:56.528315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6e105-3f79-42e9-a490-6a1f786deaf1 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:56.568328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a14c0-1e47-4ddc-9ea1-feaaff7374cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:56.569045 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a14c0-1e47-4ddc-9ea1-feaaff7374cf None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:56.569365 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a14c0-1e47-4ddc-9ea1-feaaff7374cf None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:56.569635 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a14c0-1e47-4ddc-9ea1-feaaff7374cf None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:56.570502 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a14c0-1e47-4ddc-9ea1-feaaff7374cf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:44:56.570831 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a14c0-1e47-4ddc-9ea1-feaaff7374cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:56.572246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a14c0-1e47-4ddc-9ea1-feaaff7374cf None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:56.593201 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 609/1227] 10.222.0.78 () {56 vars in 1143 bytes} [Wed May 25 14:44:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:44:56.600130 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cbc38-d484-4495-b0d2-d5b93d6c925f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:56.720236 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cbc38-d484-4495-b0d2-d5b93d6c925f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:56.759948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b9170-9112-4d06-ab5c-279eff22b445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:56.760781 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b9170-9112-4d06-ab5c-279eff22b445 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:56.761175 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b9170-9112-4d06-ab5c-279eff22b445 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:56.761759 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b9170-9112-4d06-ab5c-279eff22b445 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:56.776015 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b9170-9112-4d06-ab5c-279eff22b445 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:56.777988 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b9170-9112-4d06-ab5c-279eff22b445 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:56.802171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-e15b9170-9112-4d06-ab5c-279eff22b445 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:44:56.802171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:44:56.802171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:44:56.802171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:44:56.802171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:44:56.802171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:44:56.802171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:44:56.802171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:44:56.802171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:44:56.802171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:44:56.802171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:44:56.802171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:44:56.802171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:44:56.802171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:44:56.802171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:44:56.803320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:44:56.803320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:44:56.803320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:44:56.803320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:44:56.803320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:44:56.803320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:44:56.803320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:44:56.803320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:44:56.803320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:44:56.803320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:44:56.803320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:44:56.803320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:44:56.803320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:44:56.803320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:44:56.803320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:44:56.803320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:44:56.803320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:44:56.808997 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:44:56.808997 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:44:56.808997 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:44:56.808997 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:44:56.808997 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 619/1228] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:44:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:44:56.809426 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a14c0-1e47-4ddc-9ea1-feaaff7374cf None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:56.937415 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a14c0-1e47-4ddc-9ea1-feaaff7374cf None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:56.939965 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:56.982712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968940d-b7a8-4cd6-a0d5-51cb9c111fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:56.983577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968940d-b7a8-4cd6-a0d5-51cb9c111fa9 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:56.983854 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968940d-b7a8-4cd6-a0d5-51cb9c111fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:56.984187 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968940d-b7a8-4cd6-a0d5-51cb9c111fa9 None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:56.985037 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968940d-b7a8-4cd6-a0d5-51cb9c111fa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:44:56.985397 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968940d-b7a8-4cd6-a0d5-51cb9c111fa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:56.986917 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968940d-b7a8-4cd6-a0d5-51cb9c111fa9 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:56.999696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 610/1229] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:44:57.006688 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b9170-9112-4d06-ab5c-279eff22b445 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:57.017873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:57.126557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b9170-9112-4d06-ab5c-279eff22b445 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:57.165815 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e738c-3748-4dc2-bf4a-a05affffd57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:57.166725 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e738c-3748-4dc2-bf4a-a05affffd57f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:57.167084 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e738c-3748-4dc2-bf4a-a05affffd57f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:57.167397 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e738c-3748-4dc2-bf4a-a05affffd57f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:57.176825 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e738c-3748-4dc2-bf4a-a05affffd57f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:57.178400 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e738c-3748-4dc2-bf4a-a05affffd57f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:57.195430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-635e738c-3748-4dc2-bf4a-a05affffd57f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:44:57.195430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:44:57.195430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:44:57.195430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:44:57.195430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:44:57.195430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:44:57.195430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:44:57.195430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:44:57.195430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:44:57.195430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:44:57.195430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:44:57.195430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:44:57.195430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:44:57.195430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:44:57.195430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:44:57.197412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:44:57.197412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:44:57.197412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:44:57.197412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:44:57.197412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:44:57.197412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:44:57.197412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:44:57.197412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:44:57.197412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:44:57.197412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:44:57.197412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:44:57.197412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:44:57.197412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:44:57.197412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:44:57.197412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:44:57.197412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:44:57.197412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:44:57.199265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:44:57.199265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:44:57.199265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:44:57.199265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:44:57.199265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:44:57.199265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:44:57.199265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:44:57.199265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:44:57.199265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 620/1230] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:44:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:44:57.203858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968940d-b7a8-4cd6-a0d5-51cb9c111fa9 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:57.345228 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968940d-b7a8-4cd6-a0d5-51cb9c111fa9 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:57.385309 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96f192-353b-4e47-92f7-0846e20dcaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:57.386518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96f192-353b-4e47-92f7-0846e20dcaa9 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:57.387018 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96f192-353b-4e47-92f7-0846e20dcaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:57.387514 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96f192-353b-4e47-92f7-0846e20dcaa9 None admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:57.389054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96f192-353b-4e47-92f7-0846e20dcaa9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:44:57.389614 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96f192-353b-4e47-92f7-0846e20dcaa9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:57.392421 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96f192-353b-4e47-92f7-0846e20dcaa9 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:57.407217 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 611/1231] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:44:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:44:57.413253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e738c-3748-4dc2-bf4a-a05affffd57f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:57.488967 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:57.531093 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e738c-3748-4dc2-bf4a-a05affffd57f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:57.569112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05414131-aa42-46c2-bb5e-e0cd9c4de4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:57.570033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05414131-aa42-46c2-bb5e-e0cd9c4de4cc None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:57.570335 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05414131-aa42-46c2-bb5e-e0cd9c4de4cc None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:57.570613 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05414131-aa42-46c2-bb5e-e0cd9c4de4cc None admin] PATH_INFO: `/v3/projects/908a13d2b49544fbaf760b07e213bb7d/users/63789c0a113447ff90933d447cf5bac3/roles/336f27e7f8624a30b30cef9acada97a0` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:57.588512 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05414131-aa42-46c2-bb5e-e0cd9c4de4cc None admin] RBAC: Authorizing `identity:create_grant(project_id=908a13d2b49544fbaf760b07e213bb7d, user_id=63789c0a113447ff90933d447cf5bac3, role_id=336f27e7f8624a30b30cef9acada97a0)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:57.590092 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05414131-aa42-46c2-bb5e-e0cd9c4de4cc None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:57.611394 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 621/1232] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:44:57 2022] PUT /identity/v3/projects/908a13d2b49544fbaf760b07e213bb7d/users/63789c0a113447ff90933d447cf5bac3/roles/336f27e7f8624a30b30cef9acada97a0 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:44:59.999454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da83a4-1cd3-45ae-9ee2-f1c08b8181bf None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:00.000051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da83a4-1cd3-45ae-9ee2-f1c08b8181bf None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:00.000051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da83a4-1cd3-45ae-9ee2-f1c08b8181bf None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:00.001386 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 612/1233] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:45:00.007056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786a8ea-515e-41a4-a58a-1f8b66285c91 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:00.007604 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786a8ea-515e-41a4-a58a-1f8b66285c91 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:00.007604 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786a8ea-515e-41a4-a58a-1f8b66285c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:00.025249 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-d786a8ea-515e-41a4-a58a-1f8b66285c91 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:00.026531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d786a8ea-515e-41a4-a58a-1f8b66285c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:00.097842 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 622/1234] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:45:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:45:00.122080 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da83a4-1cd3-45ae-9ee2-f1c08b8181bf None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:00.148674 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:00.235983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da83a4-1cd3-45ae-9ee2-f1c08b8181bf None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:00.281640 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c384d-1ee4-4008-855a-8ef054c8e5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:00.283342 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c384d-1ee4-4008-855a-8ef054c8e5d2 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:00.284704 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c384d-1ee4-4008-855a-8ef054c8e5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:00.285247 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c384d-1ee4-4008-855a-8ef054c8e5d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:00.304970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c384d-1ee4-4008-855a-8ef054c8e5d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:45:00.317696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-361c384d-1ee4-4008-855a-8ef054c8e5d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:45:00.317696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:00.317696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:45:00.317696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:45:00.317696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:45:00.317696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:45:00.317696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:45:00.317696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:45:00.317696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:45:00.317696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:45:00.317696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:45:00.317696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:45:00.317696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:45:00.317696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:45:00.317696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:45:00.317696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:45:00.319885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:45:00.319885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:45:00.319885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:45:00.319885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:45:00.319885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:45:00.319885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:45:00.319885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:45:00.319885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:45:00.319885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:45:00.319885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:45:00.319885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:45:00.319885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:45:00.319885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:45:00.319885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:45:00.319885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:45:00.319885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:45:00.321876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:45:00.321876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:45:00.321876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:45:00.321876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:45:00.321876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:45:00.321876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:45:00.321876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:45:00.321876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:45:00.321876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:45:00.321876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:45:00.321876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:45:00.321876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:45:00.321876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 613/1235] 10.222.0.78 () {56 vars in 1102 bytes} [Wed May 25 14:45:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:45:00.326700 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786a8ea-515e-41a4-a58a-1f8b66285c91 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:00.378647 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:00.436944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786a8ea-515e-41a4-a58a-1f8b66285c91 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:00.475879 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e87f3d-1e86-478f-95c0-58dfdf1cfaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:00.476784 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e87f3d-1e86-478f-95c0-58dfdf1cfaa7 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:00.477196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e87f3d-1e86-478f-95c0-58dfdf1cfaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:00.477583 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e87f3d-1e86-478f-95c0-58dfdf1cfaa7 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:00.478512 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e87f3d-1e86-478f-95c0-58dfdf1cfaa7 None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:00.478975 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e87f3d-1e86-478f-95c0-58dfdf1cfaa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:00.480467 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e87f3d-1e86-478f-95c0-58dfdf1cfaa7 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:00.489212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 623/1236] 10.222.0.78 () {56 vars in 1103 bytes} [Wed May 25 14:45:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:45:00.494871 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c384d-1ee4-4008-855a-8ef054c8e5d2 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:00.589561 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:00.608495 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c384d-1ee4-4008-855a-8ef054c8e5d2 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:00.645054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c392a9-3979-4e5c-b484-3cb02dfd5699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:00.645855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c392a9-3979-4e5c-b484-3cb02dfd5699 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:00.646220 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c392a9-3979-4e5c-b484-3cb02dfd5699 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:00.646526 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c392a9-3979-4e5c-b484-3cb02dfd5699 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:00.657977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c392a9-3979-4e5c-b484-3cb02dfd5699 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:00.659990 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c392a9-3979-4e5c-b484-3cb02dfd5699 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:00.681128 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-e3c392a9-3979-4e5c-b484-3cb02dfd5699 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:45:00.681128 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:00.681128 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:45:00.681128 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:45:00.681128 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:45:00.681128 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:45:00.681128 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:45:00.681128 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:45:00.681128 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:45:00.681128 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:45:00.681128 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:45:00.681128 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:45:00.681128 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:45:00.681128 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:45:00.681128 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:45:00.682135 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:45:00.682135 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:45:00.682135 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:45:00.682135 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:45:00.682135 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:45:00.682135 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:45:00.682135 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:45:00.682135 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:45:00.682135 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:45:00.682135 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:45:00.682135 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:45:00.682135 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:45:00.682135 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:45:00.682135 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:45:00.682135 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:45:00.682135 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:45:00.682135 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:45:00.683233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:45:00.683233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:45:00.683233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:45:00.683233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:45:00.683233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 614/1237] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:45:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:45:00.688202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e87f3d-1e86-478f-95c0-58dfdf1cfaa7 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:00.797314 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e87f3d-1e86-478f-95c0-58dfdf1cfaa7 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:00.808509 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:00.837886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c49c6-047b-42dd-9e1a-800f22f793a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:00.838619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c49c6-047b-42dd-9e1a-800f22f793a5 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:00.838926 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c49c6-047b-42dd-9e1a-800f22f793a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:00.839214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c49c6-047b-42dd-9e1a-800f22f793a5 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:00.840003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448c49c6-047b-42dd-9e1a-800f22f793a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:00.840312 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448c49c6-047b-42dd-9e1a-800f22f793a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:00.841687 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448c49c6-047b-42dd-9e1a-800f22f793a5 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:00.854868 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 624/1238] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:45:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:00.861194 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c392a9-3979-4e5c-b484-3cb02dfd5699 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:00.987426 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c392a9-3979-4e5c-b484-3cb02dfd5699 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:01.027268 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95a205-00ee-4767-b418-af2f542bf197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:01.028488 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95a205-00ee-4767-b418-af2f542bf197 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:01.028770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95a205-00ee-4767-b418-af2f542bf197 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:01.029151 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95a205-00ee-4767-b418-af2f542bf197 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:01.032714 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:01.039559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95a205-00ee-4767-b418-af2f542bf197 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:01.041014 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95a205-00ee-4767-b418-af2f542bf197 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:01.056989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-ce95a205-00ee-4767-b418-af2f542bf197 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 14:45:01.056989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:01.056989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:45:01.056989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:45:01.056989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:45:01.056989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:45:01.056989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:45:01.056989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:45:01.056989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:45:01.056989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:45:01.056989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:45:01.056989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:45:01.056989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:45:01.056989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:45:01.056989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:45:01.056989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:45:01.058238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:45:01.058238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:45:01.058238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:45:01.058238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:45:01.058238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:45:01.058238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:45:01.058238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:45:01.058238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:45:01.058238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:45:01.058238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:45:01.058238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:45:01.058238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:45:01.058238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:45:01.058238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:45:01.058238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:45:01.058238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:45:01.058238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:45:01.058238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:45:01.059233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:45:01.059233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:45:01.059233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:45:01.059233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:45:01.059233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:45:01.059233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:45:01.059233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:45:01.059233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:45:01.059233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 14:45:01.059233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:45:01.059233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 615/1239] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:45:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:45:01.064161 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c49c6-047b-42dd-9e1a-800f22f793a5 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:01.195265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c49c6-047b-42dd-9e1a-800f22f793a5 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:01.235569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02ffc0-fd24-4759-aa67-199ad4cd4c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:01.236274 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02ffc0-fd24-4759-aa67-199ad4cd4c2f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:01.236540 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02ffc0-fd24-4759-aa67-199ad4cd4c2f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:01.236844 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02ffc0-fd24-4759-aa67-199ad4cd4c2f None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:01.237689 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02ffc0-fd24-4759-aa67-199ad4cd4c2f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:01.238001 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02ffc0-fd24-4759-aa67-199ad4cd4c2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:01.239486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02ffc0-fd24-4759-aa67-199ad4cd4c2f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:01.259721 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 625/1240] 10.222.0.78 () {56 vars in 1143 bytes} [Wed May 25 14:45:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:01.265491 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95a205-00ee-4767-b418-af2f542bf197 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:01.383352 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95a205-00ee-4767-b418-af2f542bf197 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:01.421665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c01ac-3652-48dd-af23-93c058a09ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:01.422452 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c01ac-3652-48dd-af23-93c058a09ae6 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:01.422804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c01ac-3652-48dd-af23-93c058a09ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:01.423151 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c01ac-3652-48dd-af23-93c058a09ae6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:01.489509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:01.497945 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c01ac-3652-48dd-af23-93c058a09ae6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:01.499423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c01ac-3652-48dd-af23-93c058a09ae6 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:01.624531 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 616/1241] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:45:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:45:01.631405 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02ffc0-fd24-4759-aa67-199ad4cd4c2f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:01.664150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:01.769078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02ffc0-fd24-4759-aa67-199ad4cd4c2f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:01.823917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aff04c-c351-4e63-9c4b-490e8fb21c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:01.824799 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aff04c-c351-4e63-9c4b-490e8fb21c49 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:01.825090 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aff04c-c351-4e63-9c4b-490e8fb21c49 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:01.825371 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aff04c-c351-4e63-9c4b-490e8fb21c49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:01.839624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aff04c-c351-4e63-9c4b-490e8fb21c49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:01.841480 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aff04c-c351-4e63-9c4b-490e8fb21c49 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:01.867660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-68aff04c-c351-4e63-9c4b-490e8fb21c49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:45:01.867660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:01.867660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:45:01.867660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:45:01.867660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:45:01.867660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:45:01.867660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:45:01.867660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:45:01.867660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:45:01.867660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:45:01.867660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:45:01.867660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:45:01.867660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:45:01.867660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:45:01.867660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:45:01.868859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:45:01.868859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:45:01.868859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:45:01.868859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:45:01.868859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:45:01.868859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:45:01.868859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:45:01.868859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:45:01.868859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:45:01.868859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:45:01.868859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:45:01.868859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:45:01.868859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:45:01.868859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:45:01.868859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:45:01.868859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:45:01.868859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:45:01.869967 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:45:01.869967 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:45:01.869967 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:45:01.869967 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:45:01.869967 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 626/1242] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:45:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:45:01.876279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c01ac-3652-48dd-af23-93c058a09ae6 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:02.001246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:02.015347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c01ac-3652-48dd-af23-93c058a09ae6 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:02.056138 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20a973-efdc-44e1-bd2a-5fa84720156a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:02.057264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20a973-efdc-44e1-bd2a-5fa84720156a None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:02.057264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20a973-efdc-44e1-bd2a-5fa84720156a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:02.057565 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20a973-efdc-44e1-bd2a-5fa84720156a None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:02.058059 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20a973-efdc-44e1-bd2a-5fa84720156a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:02.058363 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20a973-efdc-44e1-bd2a-5fa84720156a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:02.059757 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20a973-efdc-44e1-bd2a-5fa84720156a None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:02.084774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 617/1243] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:02.086073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aff04c-c351-4e63-9c4b-490e8fb21c49 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:02.175439 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:02.214621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aff04c-c351-4e63-9c4b-490e8fb21c49 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:02.254114 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e6729-47ab-468f-acec-b0219394e8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:02.254840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e6729-47ab-468f-acec-b0219394e8cb None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:02.255126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e6729-47ab-468f-acec-b0219394e8cb None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:02.255403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e6729-47ab-468f-acec-b0219394e8cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:02.264995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e6729-47ab-468f-acec-b0219394e8cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:02.266512 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e6729-47ab-468f-acec-b0219394e8cb None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:02.283128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-615e6729-47ab-468f-acec-b0219394e8cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:45:02.283128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:02.283128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:45:02.283128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:45:02.283128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:45:02.283128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:45:02.283128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:45:02.283128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:45:02.283128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:45:02.283128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:45:02.283128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:45:02.283128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:45:02.283128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:45:02.283128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:45:02.283128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:45:02.284937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:45:02.284937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:45:02.284937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:45:02.284937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:45:02.284937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:45:02.284937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:45:02.284937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:45:02.284937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:45:02.284937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:45:02.284937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:45:02.284937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:45:02.284937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:45:02.284937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:45:02.284937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:45:02.284937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:45:02.284937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:45:02.284937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:45:02.285936 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:45:02.285936 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:45:02.285936 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:45:02.285936 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:45:02.285936 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:45:02.285936 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:45:02.285936 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:45:02.285936 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:45:02.285936 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 627/1244] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:45:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:45:02.292443 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20a973-efdc-44e1-bd2a-5fa84720156a None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:02.413438 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20a973-efdc-44e1-bd2a-5fa84720156a None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:02.460009 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc9c39-7caf-47b0-bca3-7a6dc6608fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:02.461148 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc9c39-7caf-47b0-bca3-7a6dc6608fbe None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:02.461620 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc9c39-7caf-47b0-bca3-7a6dc6608fbe None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:02.462152 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc9c39-7caf-47b0-bca3-7a6dc6608fbe None admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:02.463662 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc9c39-7caf-47b0-bca3-7a6dc6608fbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:02.464212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc9c39-7caf-47b0-bca3-7a6dc6608fbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:02.466997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc9c39-7caf-47b0-bca3-7a6dc6608fbe None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:02.483440 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 618/1245] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:45:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:02.489873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e6729-47ab-468f-acec-b0219394e8cb None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:02.607913 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e6729-47ab-468f-acec-b0219394e8cb None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:02.647873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9b302-cd00-432d-8c5d-9856655fea48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:02.648526 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9b302-cd00-432d-8c5d-9856655fea48 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:02.648840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9b302-cd00-432d-8c5d-9856655fea48 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:02.649087 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9b302-cd00-432d-8c5d-9856655fea48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:02.650120 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9b302-cd00-432d-8c5d-9856655fea48 None admin] RBAC: Adding query filter params (role.id=336f27e7f8624a30b30cef9acada97a0, scope.project.id=908a13d2b49544fbaf760b07e213bb7d, user.id=63789c0a113447ff90933d447cf5bac3) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:02.650488 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9b302-cd00-432d-8c5d-9856655fea48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:02.652724 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9b302-cd00-432d-8c5d-9856655fea48 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:02.659786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 628/1246] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:45:02 2022] GET /identity/v3/role_assignments?user.id=63789c0a113447ff90933d447cf5bac3&scope.project.id=908a13d2b49544fbaf760b07e213bb7d&role.id=336f27e7f8624a30b30cef9acada97a0 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:45:05.028515 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0c603-471f-4832-aabb-d27ed13d57da None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:05.029168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0c603-471f-4832-aabb-d27ed13d57da None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:05.029168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0c603-471f-4832-aabb-d27ed13d57da None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:05.030469 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 619/1247] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:45:05.036635 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad90ccb-826f-40d9-a1df-64b121ffb92f None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:05.037269 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad90ccb-826f-40d9-a1df-64b121ffb92f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:05.037269 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad90ccb-826f-40d9-a1df-64b121ffb92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:05.042254 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:05.055474 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-4ad90ccb-826f-40d9-a1df-64b121ffb92f None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:05.056683 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4ad90ccb-826f-40d9-a1df-64b121ffb92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:05.131989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 629/1248] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:45:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:45:05.160850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0c603-471f-4832-aabb-d27ed13d57da None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:05.183940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:05.286003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0c603-471f-4832-aabb-d27ed13d57da None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:05.332201 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5c95f-cad8-46b1-8a6c-471693ba089e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:05.333015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5c95f-cad8-46b1-8a6c-471693ba089e None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:05.333311 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5c95f-cad8-46b1-8a6c-471693ba089e None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:05.333622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5c95f-cad8-46b1-8a6c-471693ba089e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:05.347329 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5c95f-cad8-46b1-8a6c-471693ba089e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:45:05.354998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-cdb5c95f-cad8-46b1-8a6c-471693ba089e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:45:05.354998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:05.354998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:45:05.354998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:45:05.354998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:45:05.354998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:45:05.354998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:45:05.354998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:45:05.354998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:45:05.354998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:45:05.354998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:45:05.354998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:45:05.354998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:45:05.354998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:45:05.354998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:45:05.354998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:45:05.356183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:45:05.356183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:45:05.356183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:45:05.356183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:45:05.356183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:45:05.356183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:45:05.356183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:45:05.356183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:45:05.356183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:45:05.356183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:45:05.356183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:45:05.356183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:45:05.356183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:45:05.356183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:45:05.356183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:45:05.356183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:45:05.357210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:45:05.357210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:45:05.357210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:45:05.357210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:45:05.357210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:45:05.357210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:45:05.357210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:45:05.357210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:45:05.357210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:45:05.357210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:45:05.357210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:45:05.357210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:45:05.357210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 620/1249] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:45:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:45:05.362752 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad90ccb-826f-40d9-a1df-64b121ffb92f None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:05.496000 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad90ccb-826f-40d9-a1df-64b121ffb92f None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:05.541132 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:05.549138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f98c3-707d-4228-a40b-45a62953d31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:05.549833 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f98c3-707d-4228-a40b-45a62953d31b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:05.550150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f98c3-707d-4228-a40b-45a62953d31b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:05.550429 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f98c3-707d-4228-a40b-45a62953d31b None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:05.551209 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f98c3-707d-4228-a40b-45a62953d31b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:05.551519 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f98c3-707d-4228-a40b-45a62953d31b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:05.553062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f98c3-707d-4228-a40b-45a62953d31b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:05.564961 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 630/1250] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:45:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:05.570967 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5c95f-cad8-46b1-8a6c-471693ba089e None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:05.676909 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:05.712452 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5c95f-cad8-46b1-8a6c-471693ba089e None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:05.784638 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbd2ed-0bcd-4449-96da-d74fa37b8848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:05.785173 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbd2ed-0bcd-4449-96da-d74fa37b8848 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:05.785450 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbd2ed-0bcd-4449-96da-d74fa37b8848 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:05.785716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbd2ed-0bcd-4449-96da-d74fa37b8848 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:05.800473 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dbd2ed-0bcd-4449-96da-d74fa37b8848 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:05.803906 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dbd2ed-0bcd-4449-96da-d74fa37b8848 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:05.826266 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-a5dbd2ed-0bcd-4449-96da-d74fa37b8848 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:45:05.826266 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:05.826266 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:45:05.826266 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:45:05.826266 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:45:05.826266 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:45:05.826266 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:45:05.826266 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:45:05.826266 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:45:05.826266 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:45:05.826266 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:45:05.826266 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:45:05.826266 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:45:05.826266 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:45:05.826266 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:45:05.827954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:45:05.827954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:45:05.827954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:45:05.827954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:45:05.827954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:45:05.827954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:45:05.827954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:45:05.827954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:45:05.827954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:45:05.827954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:45:05.827954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:45:05.827954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:45:05.827954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:45:05.827954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:45:05.827954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:45:05.827954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:45:05.827954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:45:05.829046 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:45:05.829046 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:45:05.829046 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:45:05.829046 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:45:05.829046 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 621/1251] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:45:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:45:05.834972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f98c3-707d-4228-a40b-45a62953d31b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:05.970585 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f98c3-707d-4228-a40b-45a62953d31b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:06.020097 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9fcc0b-5150-4c72-b8a6-2b5a45bdca69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:06.020815 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fcc0b-5150-4c72-b8a6-2b5a45bdca69 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:06.021099 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fcc0b-5150-4c72-b8a6-2b5a45bdca69 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:06.021376 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fcc0b-5150-4c72-b8a6-2b5a45bdca69 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:06.022130 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9fcc0b-5150-4c72-b8a6-2b5a45bdca69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:06.022437 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9fcc0b-5150-4c72-b8a6-2b5a45bdca69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:06.023833 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9fcc0b-5150-4c72-b8a6-2b5a45bdca69 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:06.037942 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 631/1252] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:06.044201 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbd2ed-0bcd-4449-96da-d74fa37b8848 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:06.154284 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbd2ed-0bcd-4449-96da-d74fa37b8848 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:06.184120 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:06.197632 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b2e739-89fe-4fc8-9ed7-f51d3302e45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:06.198487 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2e739-89fe-4fc8-9ed7-f51d3302e45b None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:06.198760 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2e739-89fe-4fc8-9ed7-f51d3302e45b None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:06.199030 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2e739-89fe-4fc8-9ed7-f51d3302e45b None admin] PATH_INFO: `/v3/users/placement` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:06.207809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b2e739-89fe-4fc8-9ed7-f51d3302e45b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:06.209255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b2e739-89fe-4fc8-9ed7-f51d3302e45b None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:06.226872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-18b2e739-89fe-4fc8-9ed7-f51d3302e45b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 14:45:06.226872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:06.226872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:45:06.226872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:45:06.226872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:45:06.226872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:45:06.226872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:45:06.226872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:45:06.226872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:45:06.226872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:45:06.226872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:45:06.226872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:45:06.226872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:45:06.226872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:45:06.226872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:45:06.226872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:45:06.228020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:45:06.228020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:45:06.228020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:45:06.228020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:45:06.228020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:45:06.228020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:45:06.228020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:45:06.228020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:45:06.228020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:45:06.228020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:45:06.228020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:45:06.228020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:45:06.228020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:45:06.228020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:45:06.228020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:45:06.228020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:45:06.228020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:45:06.228020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:45:06.229198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:45:06.229198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:45:06.229198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:45:06.229198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:45:06.229198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:45:06.229198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:45:06.229198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:45:06.229198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:45:06.229198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 14:45:06.229198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:45:06.229198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 622/1253] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:45:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:45:06.234723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9fcc0b-5150-4c72-b8a6-2b5a45bdca69 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:06.313749 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:06.367315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9fcc0b-5150-4c72-b8a6-2b5a45bdca69 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:06.407205 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd73207-74a7-424a-992c-8f33c70cd111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:06.408051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd73207-74a7-424a-992c-8f33c70cd111 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:06.408364 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd73207-74a7-424a-992c-8f33c70cd111 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:06.408651 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd73207-74a7-424a-992c-8f33c70cd111 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:06.409813 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd73207-74a7-424a-992c-8f33c70cd111 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:06.410166 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd73207-74a7-424a-992c-8f33c70cd111 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:06.411874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd73207-74a7-424a-992c-8f33c70cd111 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:06.431815 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 632/1254] 10.222.0.78 () {56 vars in 1143 bytes} [Wed May 25 14:45:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:06.438207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b2e739-89fe-4fc8-9ed7-f51d3302e45b None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:06.560937 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b2e739-89fe-4fc8-9ed7-f51d3302e45b None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:06.602883 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea3e10e-8063-481f-b35c-84b767823a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:06.603925 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3e10e-8063-481f-b35c-84b767823a81 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:06.604359 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3e10e-8063-481f-b35c-84b767823a81 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:06.604763 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3e10e-8063-481f-b35c-84b767823a81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:06.694493 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:06.710284 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea3e10e-8063-481f-b35c-84b767823a81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:06.711743 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea3e10e-8063-481f-b35c-84b767823a81 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:06.835734 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 623/1255] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:45:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:45:06.842089 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd73207-74a7-424a-992c-8f33c70cd111 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:06.975181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd73207-74a7-424a-992c-8f33c70cd111 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:06.991151 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:07.013256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54286b40-8864-4167-a4cc-68de60d34582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:07.014031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54286b40-8864-4167-a4cc-68de60d34582 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:07.014356 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54286b40-8864-4167-a4cc-68de60d34582 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:07.014632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54286b40-8864-4167-a4cc-68de60d34582 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:07.027080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54286b40-8864-4167-a4cc-68de60d34582 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:07.029073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54286b40-8864-4167-a4cc-68de60d34582 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:07.048253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-54286b40-8864-4167-a4cc-68de60d34582 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:45:07.048253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:07.048253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:45:07.048253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:45:07.048253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:45:07.048253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:45:07.048253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:45:07.048253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:45:07.048253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:45:07.048253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:45:07.048253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:45:07.048253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:45:07.048253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:45:07.048253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:45:07.048253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:45:07.049672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:45:07.049672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:45:07.049672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:45:07.049672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:45:07.049672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:45:07.049672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:45:07.049672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:45:07.049672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:45:07.049672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:45:07.049672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:45:07.049672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:45:07.049672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:45:07.049672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:45:07.049672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:45:07.049672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:45:07.049672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:45:07.049672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:45:07.050856 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:45:07.050856 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:45:07.050856 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:45:07.050856 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:45:07.050856 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 633/1256] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:45:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:45:07.057314 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea3e10e-8063-481f-b35c-84b767823a81 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:07.151901 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:07.166357 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea3e10e-8063-481f-b35c-84b767823a81 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:07.216754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530692fa-be45-4019-8490-e8151e9b7f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:07.217529 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530692fa-be45-4019-8490-e8151e9b7f9c None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:07.217932 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530692fa-be45-4019-8490-e8151e9b7f9c None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:07.218219 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530692fa-be45-4019-8490-e8151e9b7f9c None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:07.219111 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530692fa-be45-4019-8490-e8151e9b7f9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:07.219690 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530692fa-be45-4019-8490-e8151e9b7f9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:07.221770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530692fa-be45-4019-8490-e8151e9b7f9c None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:07.237030 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 624/1257] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:45:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:07.245340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54286b40-8864-4167-a4cc-68de60d34582 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:07.381092 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54286b40-8864-4167-a4cc-68de60d34582 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:07.422540 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c5a11-5443-4c97-9f55-cd005f3b1c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:07.423280 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c5a11-5443-4c97-9f55-cd005f3b1c73 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:07.423561 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c5a11-5443-4c97-9f55-cd005f3b1c73 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:07.423870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c5a11-5443-4c97-9f55-cd005f3b1c73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:07.431939 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0c5a11-5443-4c97-9f55-cd005f3b1c73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:07.433438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0c5a11-5443-4c97-9f55-cd005f3b1c73 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:07.449065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-db0c5a11-5443-4c97-9f55-cd005f3b1c73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:45:07.449065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:07.449065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:45:07.449065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:45:07.449065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:45:07.449065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:45:07.449065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:45:07.449065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:45:07.449065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:45:07.449065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:45:07.449065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:45:07.449065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:45:07.449065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:45:07.449065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:45:07.449065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:45:07.450094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:45:07.450094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:45:07.450094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:45:07.450094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:45:07.450094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:45:07.450094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:45:07.450094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:45:07.450094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:45:07.450094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:45:07.450094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:45:07.450094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:45:07.450094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:45:07.450094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:45:07.450094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:45:07.450094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:45:07.450094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:45:07.450094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:45:07.451267 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:45:07.451267 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:45:07.451267 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:45:07.451267 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:45:07.451267 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:45:07.451267 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:45:07.451267 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:45:07.451267 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:45:07.451267 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 634/1258] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:45:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:45:07.457206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530692fa-be45-4019-8490-e8151e9b7f9c None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:07.577032 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530692fa-be45-4019-8490-e8151e9b7f9c None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:07.644761 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6388fbc-2947-45bd-98e9-68ed93a4faab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:07.645442 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6388fbc-2947-45bd-98e9-68ed93a4faab None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:07.645724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6388fbc-2947-45bd-98e9-68ed93a4faab None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:07.645998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6388fbc-2947-45bd-98e9-68ed93a4faab None admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:07.646842 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6388fbc-2947-45bd-98e9-68ed93a4faab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:07.647152 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6388fbc-2947-45bd-98e9-68ed93a4faab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:07.648635 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6388fbc-2947-45bd-98e9-68ed93a4faab None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:07.666158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 625/1259] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:45:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:07.672803 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c5a11-5443-4c97-9f55-cd005f3b1c73 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:07.702323 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:07.785304 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c5a11-5443-4c97-9f55-cd005f3b1c73 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:07.830668 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f015f-3c2d-4783-a538-aeec066b0a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:07.831448 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f015f-3c2d-4783-a538-aeec066b0a6f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:07.831614 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f015f-3c2d-4783-a538-aeec066b0a6f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:07.831891 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f015f-3c2d-4783-a538-aeec066b0a6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:07.832727 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f015f-3c2d-4783-a538-aeec066b0a6f None admin] RBAC: Adding query filter params (role.id=e566b37cd92f467198b5dc40f183effc, scope.project.id=908a13d2b49544fbaf760b07e213bb7d, user.id=63789c0a113447ff90933d447cf5bac3) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:07.833067 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f015f-3c2d-4783-a538-aeec066b0a6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:07.834661 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f015f-3c2d-4783-a538-aeec066b0a6f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:07.841780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 635/1260] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:45:07 2022] GET /identity/v3/role_assignments?user.id=63789c0a113447ff90933d447cf5bac3&scope.project.id=908a13d2b49544fbaf760b07e213bb7d&role.id=e566b37cd92f467198b5dc40f183effc => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:10.070609 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b59a18-4e2c-4b26-8a0f-b8e9090ee38a None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:10.071171 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b59a18-4e2c-4b26-8a0f-b8e9090ee38a None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:10.071171 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b59a18-4e2c-4b26-8a0f-b8e9090ee38a None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:10.072531 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 626/1261] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:45:10.077879 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6d28d-f180-4a11-bbc4-cf6f2d59669c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:10.078340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6d28d-f180-4a11-bbc4-cf6f2d59669c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:10.078475 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6d28d-f180-4a11-bbc4-cf6f2d59669c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:10.096169 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-3ae6d28d-f180-4a11-bbc4-cf6f2d59669c None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:10.097374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3ae6d28d-f180-4a11-bbc4-cf6f2d59669c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:10.164082 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 636/1262] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:45:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:45:10.185972 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b59a18-4e2c-4b26-8a0f-b8e9090ee38a None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:10.196909 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:10.299014 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b59a18-4e2c-4b26-8a0f-b8e9090ee38a None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:10.343423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e513a-0fc0-4105-b240-87504da578f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:10.344215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e513a-0fc0-4105-b240-87504da578f5 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:10.344489 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e513a-0fc0-4105-b240-87504da578f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:10.344761 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e513a-0fc0-4105-b240-87504da578f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:10.356459 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e513a-0fc0-4105-b240-87504da578f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:45:10.364653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-ba2e513a-0fc0-4105-b240-87504da578f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:45:10.364653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:10.364653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:45:10.364653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:45:10.364653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:45:10.364653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:45:10.364653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:45:10.364653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:45:10.364653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:45:10.364653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:45:10.364653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:45:10.364653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:45:10.364653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:45:10.364653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:45:10.364653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:45:10.364653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:45:10.365963 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:45:10.365963 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:45:10.365963 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:45:10.365963 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:45:10.365963 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:45:10.365963 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:45:10.365963 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:45:10.365963 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:45:10.365963 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:45:10.365963 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:45:10.365963 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:45:10.365963 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:45:10.365963 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:45:10.365963 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:45:10.365963 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:45:10.365963 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:45:10.367044 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:45:10.367044 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:45:10.367044 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:45:10.367044 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:45:10.367044 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:45:10.367044 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:45:10.367044 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:45:10.367044 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:45:10.367044 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:45:10.367044 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:45:10.367044 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:45:10.367044 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:45:10.367044 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 627/1263] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:45:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:45:10.372243 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6d28d-f180-4a11-bbc4-cf6f2d59669c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:10.390182 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:10.469820 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6d28d-f180-4a11-bbc4-cf6f2d59669c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:10.503796 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96480cce-5e2f-4fac-903f-d2bde03214e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:10.504464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96480cce-5e2f-4fac-903f-d2bde03214e8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:10.504748 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96480cce-5e2f-4fac-903f-d2bde03214e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:10.505024 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96480cce-5e2f-4fac-903f-d2bde03214e8 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:10.505766 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96480cce-5e2f-4fac-903f-d2bde03214e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:10.506101 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96480cce-5e2f-4fac-903f-d2bde03214e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:10.507480 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96480cce-5e2f-4fac-903f-d2bde03214e8 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:10.515655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 637/1264] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:45:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:10.521678 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e513a-0fc0-4105-b240-87504da578f5 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:10.588113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:10.622875 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e513a-0fc0-4105-b240-87504da578f5 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:10.656175 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f32322-a599-4578-b947-1868b8dc0ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:10.656961 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f32322-a599-4578-b947-1868b8dc0ba8 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:10.657246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f32322-a599-4578-b947-1868b8dc0ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:10.657523 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f32322-a599-4578-b947-1868b8dc0ba8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:10.668977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f32322-a599-4578-b947-1868b8dc0ba8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:10.670467 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f32322-a599-4578-b947-1868b8dc0ba8 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:10.687418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-c8f32322-a599-4578-b947-1868b8dc0ba8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:45:10.687418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:10.687418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:45:10.687418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:45:10.687418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:45:10.687418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:45:10.687418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:45:10.687418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:45:10.687418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:45:10.687418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:45:10.687418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:45:10.687418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:45:10.687418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:45:10.687418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:45:10.687418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:45:10.688497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:45:10.688497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:45:10.688497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:45:10.688497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:45:10.688497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:45:10.688497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:45:10.688497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:45:10.688497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:45:10.688497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:45:10.688497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:45:10.688497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:45:10.688497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:45:10.688497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:45:10.688497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:45:10.688497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:45:10.688497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:45:10.688497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:45:10.689605 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:45:10.689605 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:45:10.689605 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:45:10.689605 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:45:10.689605 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 628/1265] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:45:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:45:10.694472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96480cce-5e2f-4fac-903f-d2bde03214e8 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:10.775345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:10.798903 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96480cce-5e2f-4fac-903f-d2bde03214e8 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:10.833170 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d19cf-40b7-4307-b8a1-63899ba9a6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:10.833837 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d19cf-40b7-4307-b8a1-63899ba9a6be None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:10.834143 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d19cf-40b7-4307-b8a1-63899ba9a6be None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:10.834417 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d19cf-40b7-4307-b8a1-63899ba9a6be None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:10.835130 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d19cf-40b7-4307-b8a1-63899ba9a6be None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:10.835437 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d19cf-40b7-4307-b8a1-63899ba9a6be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:10.836831 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d19cf-40b7-4307-b8a1-63899ba9a6be None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:10.849729 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 638/1266] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:10.855464 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f32322-a599-4578-b947-1868b8dc0ba8 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:10.966871 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f32322-a599-4578-b947-1868b8dc0ba8 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:10.988827 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:11.003354 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efcfa4a-c257-4e02-94c4-cfdb308e02f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:11.004148 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcfa4a-c257-4e02-94c4-cfdb308e02f5 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:11.004424 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcfa4a-c257-4e02-94c4-cfdb308e02f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:11.004692 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcfa4a-c257-4e02-94c4-cfdb308e02f5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:11.013022 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efcfa4a-c257-4e02-94c4-cfdb308e02f5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:11.014495 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efcfa4a-c257-4e02-94c4-cfdb308e02f5 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:11.029538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-3efcfa4a-c257-4e02-94c4-cfdb308e02f5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 14:45:11.029538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:11.029538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:45:11.029538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:45:11.029538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:45:11.029538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:45:11.029538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:45:11.029538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:45:11.029538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:45:11.029538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:45:11.029538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:45:11.029538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:45:11.029538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:45:11.029538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:45:11.029538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:45:11.029538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:45:11.031064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:45:11.031064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:45:11.031064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:45:11.031064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:45:11.031064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:45:11.031064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:45:11.031064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:45:11.031064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:45:11.031064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:45:11.031064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:45:11.031064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:45:11.031064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:45:11.031064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:45:11.031064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:45:11.031064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:45:11.031064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:45:11.031064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:45:11.031064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:45:11.032449 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:45:11.032449 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:45:11.032449 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:45:11.032449 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:45:11.032449 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:45:11.032449 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:45:11.032449 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:45:11.032449 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:45:11.032449 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 14:45:11.032449 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:45:11.032449 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 629/1267] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:45:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:45:11.036761 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d19cf-40b7-4307-b8a1-63899ba9a6be None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:11.144232 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d19cf-40b7-4307-b8a1-63899ba9a6be None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:11.178924 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781cda9a-7579-4df9-b18c-3c3b60bf8964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:11.179607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cda9a-7579-4df9-b18c-3c3b60bf8964 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:11.179894 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cda9a-7579-4df9-b18c-3c3b60bf8964 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:11.180175 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cda9a-7579-4df9-b18c-3c3b60bf8964 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:11.181022 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781cda9a-7579-4df9-b18c-3c3b60bf8964 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:11.181334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781cda9a-7579-4df9-b18c-3c3b60bf8964 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:11.182828 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781cda9a-7579-4df9-b18c-3c3b60bf8964 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:11.192993 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:11.201637 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 639/1268] 10.222.0.78 () {56 vars in 1143 bytes} [Wed May 25 14:45:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:11.207995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efcfa4a-c257-4e02-94c4-cfdb308e02f5 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:11.318361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efcfa4a-c257-4e02-94c4-cfdb308e02f5 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:11.352762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90f67c-d098-4bd2-b731-9fc1fdb6d469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:11.353545 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90f67c-d098-4bd2-b731-9fc1fdb6d469 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:11.353816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90f67c-d098-4bd2-b731-9fc1fdb6d469 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:11.354111 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90f67c-d098-4bd2-b731-9fc1fdb6d469 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:11.364674 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90f67c-d098-4bd2-b731-9fc1fdb6d469 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:11.366132 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90f67c-d098-4bd2-b731-9fc1fdb6d469 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:11.383019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-1b90f67c-d098-4bd2-b731-9fc1fdb6d469 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:45:11.383019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:11.383019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:45:11.383019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:45:11.383019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:45:11.383019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:45:11.383019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:45:11.383019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:45:11.383019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:45:11.383019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:45:11.383019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:45:11.383019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:45:11.383019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:45:11.383019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:45:11.383019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:45:11.383940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:45:11.383940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:45:11.383940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:45:11.383940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:45:11.383940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:45:11.383940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:45:11.383940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:45:11.383940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:45:11.383940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:45:11.383940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:45:11.383940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:45:11.383940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:45:11.383940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:45:11.383940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:45:11.383940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:45:11.383940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:45:11.383940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:45:11.384846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:45:11.384846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:45:11.384846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:45:11.384846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:45:11.384846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 630/1269] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:45:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:45:11.390323 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781cda9a-7579-4df9-b18c-3c3b60bf8964 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:11.495090 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781cda9a-7579-4df9-b18c-3c3b60bf8964 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:11.530313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b909098-c0b8-40e6-894f-cc4d8c558bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:11.530986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b909098-c0b8-40e6-894f-cc4d8c558bda None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:11.531274 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b909098-c0b8-40e6-894f-cc4d8c558bda None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:11.531554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b909098-c0b8-40e6-894f-cc4d8c558bda None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:11.532274 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b909098-c0b8-40e6-894f-cc4d8c558bda None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:11.532582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b909098-c0b8-40e6-894f-cc4d8c558bda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:11.534620 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b909098-c0b8-40e6-894f-cc4d8c558bda None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:11.546205 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 640/1270] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:11.552230 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90f67c-d098-4bd2-b731-9fc1fdb6d469 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:11.585086 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:11.654497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90f67c-d098-4bd2-b731-9fc1fdb6d469 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:11.687892 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39159cb7-be71-4231-9c82-4b99fcf82bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:11.688678 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39159cb7-be71-4231-9c82-4b99fcf82bf6 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:11.688961 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39159cb7-be71-4231-9c82-4b99fcf82bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:11.689267 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39159cb7-be71-4231-9c82-4b99fcf82bf6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:11.696015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39159cb7-be71-4231-9c82-4b99fcf82bf6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:11.697469 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39159cb7-be71-4231-9c82-4b99fcf82bf6 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:11.712088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-39159cb7-be71-4231-9c82-4b99fcf82bf6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:45:11.712088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:11.712088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:45:11.712088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:45:11.712088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:45:11.712088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:45:11.712088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:45:11.712088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:45:11.712088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:45:11.712088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:45:11.712088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:45:11.712088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:45:11.712088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:45:11.712088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:45:11.712088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:45:11.713834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:45:11.713834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:45:11.713834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:45:11.713834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:45:11.713834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:45:11.713834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:45:11.713834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:45:11.713834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:45:11.713834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:45:11.713834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:45:11.713834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:45:11.713834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:45:11.713834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:45:11.713834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:45:11.713834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:45:11.713834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:45:11.713834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:45:11.715953 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:45:11.715953 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:45:11.715953 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:45:11.715953 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:45:11.715953 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:45:11.715953 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:45:11.715953 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:45:11.715953 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:45:11.715953 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 631/1271] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:45:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:45:11.721228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b909098-c0b8-40e6-894f-cc4d8c558bda None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:11.781230 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:11.824237 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b909098-c0b8-40e6-894f-cc4d8c558bda None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:11.858645 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4a8f7-7f98-4bc3-a919-9246c2d78fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:11.859231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4a8f7-7f98-4bc3-a919-9246c2d78fcd None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:11.859514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4a8f7-7f98-4bc3-a919-9246c2d78fcd None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:11.859797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4a8f7-7f98-4bc3-a919-9246c2d78fcd None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:11.860580 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4a8f7-7f98-4bc3-a919-9246c2d78fcd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:11.860892 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4a8f7-7f98-4bc3-a919-9246c2d78fcd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:11.862351 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4a8f7-7f98-4bc3-a919-9246c2d78fcd None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:11.874335 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 641/1272] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:45:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:11.880825 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39159cb7-be71-4231-9c82-4b99fcf82bf6 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:11.973669 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:11.983804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39159cb7-be71-4231-9c82-4b99fcf82bf6 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:12.019191 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4526b-1a1e-4f1b-b579-60de9a68242e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:12.020080 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4526b-1a1e-4f1b-b579-60de9a68242e None admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:12.020375 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4526b-1a1e-4f1b-b579-60de9a68242e None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:12.020630 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4526b-1a1e-4f1b-b579-60de9a68242e None admin] PATH_INFO: `/v3/projects/908a13d2b49544fbaf760b07e213bb7d/users/63789c0a113447ff90933d447cf5bac3/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:12.022698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4526b-1a1e-4f1b-b579-60de9a68242e None admin] RBAC: Authorizing `identity:create_grant(project_id=908a13d2b49544fbaf760b07e213bb7d, user_id=63789c0a113447ff90933d447cf5bac3, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:12.024145 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4526b-1a1e-4f1b-b579-60de9a68242e None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:12.038080 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 632/1273] 10.222.0.78 () {58 vars in 1439 bytes} [Wed May 25 14:45:11 2022] PUT /identity/v3/projects/908a13d2b49544fbaf760b07e213bb7d/users/63789c0a113447ff90933d447cf5bac3/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:45:14.245954 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1bd00-8d8c-4ba5-b30a-3c96dd7344ad None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:14.246576 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1bd00-8d8c-4ba5-b30a-3c96dd7344ad None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:14.246576 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1bd00-8d8c-4ba5-b30a-3c96dd7344ad None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:14.247932 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 642/1274] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:45:14.253658 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185e551-d695-4117-97d0-d6bfb6c302fa None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:14.254117 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185e551-d695-4117-97d0-d6bfb6c302fa None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:14.254263 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185e551-d695-4117-97d0-d6bfb6c302fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:14.271432 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-5185e551-d695-4117-97d0-d6bfb6c302fa None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:14.272639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-5185e551-d695-4117-97d0-d6bfb6c302fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:14.347838 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 633/1275] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:45:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:45:14.376426 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1bd00-8d8c-4ba5-b30a-3c96dd7344ad None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:14.475148 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1bd00-8d8c-4ba5-b30a-3c96dd7344ad None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:14.492147 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:14.509215 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1ea9f-2a5d-4736-8bc7-cf80d0244783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:14.509954 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1ea9f-2a5d-4736-8bc7-cf80d0244783 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:14.510263 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1ea9f-2a5d-4736-8bc7-cf80d0244783 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:14.510535 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1ea9f-2a5d-4736-8bc7-cf80d0244783 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:14.520318 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb1ea9f-2a5d-4736-8bc7-cf80d0244783 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:45:14.527984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-8bb1ea9f-2a5d-4736-8bc7-cf80d0244783 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:45:14.527984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:14.527984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:45:14.527984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:45:14.527984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:45:14.527984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:45:14.527984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:45:14.527984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:45:14.527984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:45:14.527984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:45:14.527984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:45:14.527984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:45:14.527984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:45:14.527984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:45:14.527984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:45:14.527984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:45:14.529078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:45:14.529078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:45:14.529078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:45:14.529078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:45:14.529078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:45:14.529078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:45:14.529078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:45:14.529078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:45:14.529078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:45:14.529078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:45:14.529078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:45:14.529078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:45:14.529078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:45:14.529078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:45:14.529078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:45:14.529078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:45:14.530074 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:45:14.530074 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:45:14.530074 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:45:14.530074 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:45:14.530074 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:45:14.530074 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:45:14.530074 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:45:14.530074 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:45:14.530074 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:45:14.530074 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:45:14.530074 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:45:14.530074 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:45:14.530074 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 643/1276] 10.222.0.78 () {56 vars in 1096 bytes} [Wed May 25 14:45:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:45:14.535133 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185e551-d695-4117-97d0-d6bfb6c302fa None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:14.616073 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:14.635413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185e551-d695-4117-97d0-d6bfb6c302fa None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:14.669239 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6b831-d402-46e5-8344-e435e0e1ec67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:14.669899 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6b831-d402-46e5-8344-e435e0e1ec67 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:14.670212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6b831-d402-46e5-8344-e435e0e1ec67 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:14.670492 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6b831-d402-46e5-8344-e435e0e1ec67 None admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:14.671229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae6b831-d402-46e5-8344-e435e0e1ec67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:14.671529 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae6b831-d402-46e5-8344-e435e0e1ec67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:14.672900 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae6b831-d402-46e5-8344-e435e0e1ec67 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:14.681305 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 634/1277] 10.222.0.78 () {56 vars in 1099 bytes} [Wed May 25 14:45:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:14.687069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1ea9f-2a5d-4736-8bc7-cf80d0244783 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:14.786852 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1ea9f-2a5d-4736-8bc7-cf80d0244783 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:14.819823 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2cfb98-d092-485f-80f1-ea4d87032465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:14.820612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cfb98-d092-485f-80f1-ea4d87032465 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:14.820898 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cfb98-d092-485f-80f1-ea4d87032465 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:14.821176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2cfb98-d092-485f-80f1-ea4d87032465 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:14.833011 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2cfb98-d092-485f-80f1-ea4d87032465 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:14.834532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2cfb98-d092-485f-80f1-ea4d87032465 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:14.852206 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-ae2cfb98-d092-485f-80f1-ea4d87032465 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:45:14.852206 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:14.852206 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:45:14.852206 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:45:14.852206 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:45:14.852206 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:45:14.852206 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:45:14.852206 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:45:14.852206 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:45:14.852206 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:45:14.852206 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:45:14.852206 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:45:14.852206 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:45:14.852206 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:45:14.852206 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:45:14.853272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:45:14.853272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:45:14.853272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:45:14.853272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:45:14.853272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:45:14.853272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:45:14.853272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:45:14.853272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:45:14.853272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:45:14.853272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:45:14.853272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:45:14.853272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:45:14.853272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:45:14.853272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:45:14.853272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:45:14.853272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:45:14.853272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:45:14.854222 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:45:14.854222 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:45:14.854222 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:45:14.854222 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:45:14.854222 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 644/1278] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:45:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:45:14.859099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6b831-d402-46e5-8344-e435e0e1ec67 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:14.957915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6b831-d402-46e5-8344-e435e0e1ec67 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:14.990059 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e48294-962c-4336-bd9d-cb285ea58378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:14.990715 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e48294-962c-4336-bd9d-cb285ea58378 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:14.990997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e48294-962c-4336-bd9d-cb285ea58378 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:14.991273 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e48294-962c-4336-bd9d-cb285ea58378 None admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:14.991987 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e48294-962c-4336-bd9d-cb285ea58378 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:14.992294 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e48294-962c-4336-bd9d-cb285ea58378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:14.993657 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e48294-962c-4336-bd9d-cb285ea58378 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:15.005791 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 635/1279] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:15.011590 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2cfb98-d092-485f-80f1-ea4d87032465 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:15.042681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:15.112217 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2cfb98-d092-485f-80f1-ea4d87032465 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:15.146201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d88fe1-b295-4d1f-8b63-030a109dd83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:15.146913 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d88fe1-b295-4d1f-8b63-030a109dd83f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:15.147200 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d88fe1-b295-4d1f-8b63-030a109dd83f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:15.147483 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d88fe1-b295-4d1f-8b63-030a109dd83f None admin] PATH_INFO: `/v3/users/placement` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:15.155243 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d88fe1-b295-4d1f-8b63-030a109dd83f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:15.156697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d88fe1-b295-4d1f-8b63-030a109dd83f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:15.170883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-a5d88fe1-b295-4d1f-8b63-030a109dd83f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 14:45:15.170883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:15.170883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:45:15.170883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:45:15.170883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:45:15.170883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:45:15.170883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:45:15.170883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:45:15.170883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:45:15.170883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:45:15.170883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:45:15.170883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:45:15.170883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:45:15.170883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:45:15.170883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:45:15.170883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:45:15.172069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:45:15.172069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:45:15.172069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:45:15.172069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:45:15.172069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:45:15.172069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:45:15.172069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:45:15.172069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:45:15.172069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:45:15.172069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:45:15.172069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:45:15.172069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:45:15.172069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:45:15.172069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:45:15.172069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:45:15.172069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:45:15.172069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:45:15.172069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:45:15.173256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:45:15.173256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:45:15.173256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:45:15.173256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:45:15.173256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:45:15.173256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:45:15.173256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:45:15.173256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:45:15.173256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 14:45:15.173256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:45:15.173256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 645/1280] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:45:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:45:15.178690 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e48294-962c-4336-bd9d-cb285ea58378 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:15.182571 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:15.279384 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e48294-962c-4336-bd9d-cb285ea58378 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:15.317443 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2a331-535c-4675-8db1-bb832f2c6c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:15.318054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2a331-535c-4675-8db1-bb832f2c6c83 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:15.318340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2a331-535c-4675-8db1-bb832f2c6c83 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:15.318616 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2a331-535c-4675-8db1-bb832f2c6c83 None admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:15.319449 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a2a331-535c-4675-8db1-bb832f2c6c83 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:15.319758 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a2a331-535c-4675-8db1-bb832f2c6c83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:15.321178 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a2a331-535c-4675-8db1-bb832f2c6c83 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:15.338192 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 636/1281] 10.222.0.78 () {56 vars in 1143 bytes} [Wed May 25 14:45:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:15.344007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d88fe1-b295-4d1f-8b63-030a109dd83f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:15.440246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:15.450611 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d88fe1-b295-4d1f-8b63-030a109dd83f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:15.486032 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236bcdca-92ae-4b9b-8964-c00e146a0f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:15.486710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bcdca-92ae-4b9b-8964-c00e146a0f86 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:15.487049 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bcdca-92ae-4b9b-8964-c00e146a0f86 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:15.487280 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bcdca-92ae-4b9b-8964-c00e146a0f86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:15.555527 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236bcdca-92ae-4b9b-8964-c00e146a0f86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:15.556992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236bcdca-92ae-4b9b-8964-c00e146a0f86 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:15.644906 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:15.655473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 646/1282] 10.222.0.78 () {58 vars in 1282 bytes} [Wed May 25 14:45:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:45:15.661313 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2a331-535c-4675-8db1-bb832f2c6c83 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:15.721053 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:15.759827 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2a331-535c-4675-8db1-bb832f2c6c83 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:15.793638 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d130b7-82b6-46e5-b87c-76a1c295469a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:15.794454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d130b7-82b6-46e5-b87c-76a1c295469a None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:15.794740 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d130b7-82b6-46e5-b87c-76a1c295469a None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:15.795017 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d130b7-82b6-46e5-b87c-76a1c295469a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:15.804912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d130b7-82b6-46e5-b87c-76a1c295469a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:15.806378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d130b7-82b6-46e5-b87c-76a1c295469a None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:15.823913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-d9d130b7-82b6-46e5-b87c-76a1c295469a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:45:15.823913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:15.823913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:45:15.823913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:45:15.823913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:45:15.823913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:45:15.823913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:45:15.823913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:45:15.823913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:45:15.823913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:45:15.823913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:45:15.823913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:45:15.823913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:45:15.823913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:45:15.823913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:45:15.824976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:45:15.824976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:45:15.824976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:45:15.824976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:45:15.824976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:45:15.824976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:45:15.824976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:45:15.824976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:45:15.824976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:45:15.824976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:45:15.824976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:45:15.824976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:45:15.824976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:45:15.824976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:45:15.824976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:45:15.824976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:45:15.824976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:45:15.825936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:45:15.825936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:45:15.825936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:45:15.825936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:45:15.825936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 637/1283] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:45:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:45:15.831341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236bcdca-92ae-4b9b-8964-c00e146a0f86 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:15.928085 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236bcdca-92ae-4b9b-8964-c00e146a0f86 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:15.961391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5f943-63b5-441b-ae58-3bac7d8615aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:15.962059 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5f943-63b5-441b-ae58-3bac7d8615aa None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:15.962342 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5f943-63b5-441b-ae58-3bac7d8615aa None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:15.962617 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5f943-63b5-441b-ae58-3bac7d8615aa None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:15.963327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5f943-63b5-441b-ae58-3bac7d8615aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:15.963626 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5f943-63b5-441b-ae58-3bac7d8615aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:15.964997 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5f943-63b5-441b-ae58-3bac7d8615aa None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:15.975770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 647/1284] 10.222.0.78 () {56 vars in 1109 bytes} [Wed May 25 14:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:15.981367 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d130b7-82b6-46e5-b87c-76a1c295469a None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:16.079985 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d130b7-82b6-46e5-b87c-76a1c295469a None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:16.093482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:16.113680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405f98e-4ea2-4236-bba6-940f989b96e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:16.114486 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405f98e-4ea2-4236-bba6-940f989b96e1 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:16.114767 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405f98e-4ea2-4236-bba6-940f989b96e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:16.115042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405f98e-4ea2-4236-bba6-940f989b96e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:16.121582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d405f98e-4ea2-4236-bba6-940f989b96e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:16.123037 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d405f98e-4ea2-4236-bba6-940f989b96e1 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:16.136897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application [None req-d405f98e-4ea2-4236-bba6-940f989b96e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:45:16.136897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:16.136897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:45:16.136897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:45:16.136897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:45:16.136897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:45:16.136897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:45:16.136897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:45:16.136897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:45:16.136897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:45:16.136897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:45:16.136897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:45:16.136897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:45:16.136897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:45:16.136897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:45:16.138000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:45:16.138000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:45:16.138000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:45:16.138000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:45:16.138000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:45:16.138000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:45:16.138000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:45:16.138000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:45:16.138000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application with Lock( May 25 14:45:16.138000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:45:16.138000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._enter() May 25 14:45:16.138000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:45:16.138000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:45:16.138000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:45:16.138000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.creator() May 25 14:45:16.138000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:45:16.138000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application created_value = creator( May 25 14:45:16.138968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:45:16.138968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:45:16.138968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:45:16.138968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:45:16.138968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:45:16.138968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:45:16.138968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:45:16.138968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: ERROR keystone.server.flask.application May 25 14:45:16.138968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 638/1285] 10.222.0.78 () {56 vars in 1111 bytes} [Wed May 25 14:45:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:45:16.144637 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5f943-63b5-441b-ae58-3bac7d8615aa None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:16.194425 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:16.245510 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5f943-63b5-441b-ae58-3bac7d8615aa None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:16.280627 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b8df7-a39a-491c-a539-6b62a38cd233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:16.281381 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b8df7-a39a-491c-a539-6b62a38cd233 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:16.281758 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b8df7-a39a-491c-a539-6b62a38cd233 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:16.282160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b8df7-a39a-491c-a539-6b62a38cd233 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:16.283118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363b8df7-a39a-491c-a539-6b62a38cd233 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:16.283535 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363b8df7-a39a-491c-a539-6b62a38cd233 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:16.285366 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363b8df7-a39a-491c-a539-6b62a38cd233 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:16.300545 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 648/1286] 10.222.0.78 () {56 vars in 1148 bytes} [Wed May 25 14:45:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:16.306900 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405f98e-4ea2-4236-bba6-940f989b96e1 None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:16.407281 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405f98e-4ea2-4236-bba6-940f989b96e1 None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:16.439651 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a01b6-9786-4cb0-ab55-23c792e2327e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:16.440296 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a01b6-9786-4cb0-ab55-23c792e2327e None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:16.440572 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a01b6-9786-4cb0-ab55-23c792e2327e None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:16.440841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a01b6-9786-4cb0-ab55-23c792e2327e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:16.441662 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a01b6-9786-4cb0-ab55-23c792e2327e None admin] RBAC: Adding query filter params (role.id=e566b37cd92f467198b5dc40f183effc, scope.project.id=908a13d2b49544fbaf760b07e213bb7d, user.id=63789c0a113447ff90933d447cf5bac3) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:16.441967 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a01b6-9786-4cb0-ab55-23c792e2327e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:16.443422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a01b6-9786-4cb0-ab55-23c792e2327e None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:16.449158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 639/1287] 10.222.0.78 () {56 vars in 1374 bytes} [Wed May 25 14:45:16 2022] GET /identity/v3/role_assignments?user.id=63789c0a113447ff90933d447cf5bac3&scope.project.id=908a13d2b49544fbaf760b07e213bb7d&role.id=e566b37cd92f467198b5dc40f183effc => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:18.729505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfa3d8-e36e-4e00-8e83-519a30cddf4f None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:18.730098 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfa3d8-e36e-4e00-8e83-519a30cddf4f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:18.730098 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfa3d8-e36e-4e00-8e83-519a30cddf4f None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:18.731488 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 649/1288] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:45:18.736643 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e152e2-8248-47e7-94db-3162b8891c65 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:18.737014 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e152e2-8248-47e7-94db-3162b8891c65 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:18.737206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e152e2-8248-47e7-94db-3162b8891c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:18.755601 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-52e152e2-8248-47e7-94db-3162b8891c65 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:18.756817 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-52e152e2-8248-47e7-94db-3162b8891c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:18.790406 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:18.831834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 640/1289] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:45:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:45:18.855029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfa3d8-e36e-4e00-8e83-519a30cddf4f None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:18.965586 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfa3d8-e36e-4e00-8e83-519a30cddf4f None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:18.976444 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:19.001019 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8665f1-7cc8-45fa-add9-45a16e0fc9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:19.001832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8665f1-7cc8-45fa-add9-45a16e0fc9e3 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:19.002154 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8665f1-7cc8-45fa-add9-45a16e0fc9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:19.002433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8665f1-7cc8-45fa-add9-45a16e0fc9e3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:19.003092 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8665f1-7cc8-45fa-add9-45a16e0fc9e3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:19.004481 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8665f1-7cc8-45fa-add9-45a16e0fc9e3 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:19.016500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-ad8665f1-7cc8-45fa-add9-45a16e0fc9e3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:45:19.016500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:19.016500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:45:19.016500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:45:19.016500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:45:19.016500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:45:19.016500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:45:19.016500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:45:19.016500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:45:19.016500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:45:19.016500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:45:19.016500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:45:19.016500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:19.016500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:45:19.016500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:45:19.016500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:45:19.016500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:45:19.018349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:45:19.018349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:45:19.018349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:45:19.018349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:45:19.018349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:45:19.018349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:45:19.018349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:45:19.018349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:45:19.018349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:45:19.018349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:45:19.018349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:45:19.018349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:45:19.018349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:45:19.018349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:45:19.018349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:45:19.018349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:45:19.019562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:45:19.019562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:45:19.019562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:45:19.019562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:45:19.019562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:45:19.019562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:45:19.019562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:45:19.019562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:45:19.019562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:45:19.019562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:45:19.019562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:45:19.019562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:45:19.019562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:45:19.019562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:45:19.019562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 650/1290] 10.222.0.78 () {56 vars in 1117 bytes} [Wed May 25 14:45:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:45:19.024372 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e152e2-8248-47e7-94db-3162b8891c65 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:19.128411 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e152e2-8248-47e7-94db-3162b8891c65 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:19.163153 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3abc2a4-dd9a-4fab-94bd-80d9adfe9571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:19.163816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abc2a4-dd9a-4fab-94bd-80d9adfe9571 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:19.164093 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abc2a4-dd9a-4fab-94bd-80d9adfe9571 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:19.164363 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abc2a4-dd9a-4fab-94bd-80d9adfe9571 None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:19.165067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3abc2a4-dd9a-4fab-94bd-80d9adfe9571 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:19.165368 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3abc2a4-dd9a-4fab-94bd-80d9adfe9571 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:19.166766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3abc2a4-dd9a-4fab-94bd-80d9adfe9571 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:19.171805 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 641/1291] 10.222.0.78 () {56 vars in 1116 bytes} [Wed May 25 14:45:19 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:19.177698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8665f1-7cc8-45fa-add9-45a16e0fc9e3 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:19.282511 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8665f1-7cc8-45fa-add9-45a16e0fc9e3 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:19.325691 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32188de9-c1a4-4b62-9f3f-6fa87d41ecbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:19.326397 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32188de9-c1a4-4b62-9f3f-6fa87d41ecbd None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:19.326672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32188de9-c1a4-4b62-9f3f-6fa87d41ecbd None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:19.326949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32188de9-c1a4-4b62-9f3f-6fa87d41ecbd None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:19.327654 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32188de9-c1a4-4b62-9f3f-6fa87d41ecbd None admin] RBAC: Adding query filter params (type=placement) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:19.328051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32188de9-c1a4-4b62-9f3f-6fa87d41ecbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:19.329367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32188de9-c1a4-4b62-9f3f-6fa87d41ecbd None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:19.336436 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 651/1292] 10.222.0.78 () {56 vars in 1116 bytes} [Wed May 25 14:45:19 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:21.593092 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1c5cd-9253-4c6a-8318-e579044db36d None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:21.593684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1c5cd-9253-4c6a-8318-e579044db36d None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:21.593684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1c5cd-9253-4c6a-8318-e579044db36d None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:21.595031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 642/1293] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:45:21.600548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89a29e-e0df-4a82-a06f-51cce69c6768 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:21.601009 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89a29e-e0df-4a82-a06f-51cce69c6768 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:21.601113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89a29e-e0df-4a82-a06f-51cce69c6768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:21.617597 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-df89a29e-e0df-4a82-a06f-51cce69c6768 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:21.618824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-df89a29e-e0df-4a82-a06f-51cce69c6768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:21.648314 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:21.684813 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 652/1294] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:45:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:45:21.708295 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1c5cd-9253-4c6a-8318-e579044db36d None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:21.738683 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:21.808892 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1c5cd-9253-4c6a-8318-e579044db36d None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:21.843527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4add99-e219-4e5d-b507-b69999d819c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:21.844088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4add99-e219-4e5d-b507-b69999d819c8 None admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:21.844364 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4add99-e219-4e5d-b507-b69999d819c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:21.844630 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4add99-e219-4e5d-b507-b69999d819c8 None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:21.845378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4add99-e219-4e5d-b507-b69999d819c8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:21.846796 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4add99-e219-4e5d-b507-b69999d819c8 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:21.859937 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 643/1295] 10.222.0.78 () {60 vars in 1141 bytes} [Wed May 25 14:45:21 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:45:24.092764 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a9130-aafb-4068-8646-f59bc9361a9a None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:24.093354 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a9130-aafb-4068-8646-f59bc9361a9a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:24.093354 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a9130-aafb-4068-8646-f59bc9361a9a None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:24.094716 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 653/1296] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:45:24.100031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc072744-d5b7-40db-99b7-f829effe34bf None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:24.100466 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc072744-d5b7-40db-99b7-f829effe34bf None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:24.100565 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc072744-d5b7-40db-99b7-f829effe34bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:24.117701 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-fc072744-d5b7-40db-99b7-f829effe34bf None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:24.118911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-fc072744-d5b7-40db-99b7-f829effe34bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:24.194970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 644/1297] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:45:24 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:45:24.218422 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a9130-aafb-4068-8646-f59bc9361a9a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:24.332517 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a9130-aafb-4068-8646-f59bc9361a9a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:24.375326 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd07a23-9eef-4614-95dd-6ba28597ff77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:24.376192 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd07a23-9eef-4614-95dd-6ba28597ff77 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:24.376551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd07a23-9eef-4614-95dd-6ba28597ff77 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:24.376840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd07a23-9eef-4614-95dd-6ba28597ff77 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:24.378134 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd07a23-9eef-4614-95dd-6ba28597ff77 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:24.379585 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd07a23-9eef-4614-95dd-6ba28597ff77 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:24.392062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-1cd07a23-9eef-4614-95dd-6ba28597ff77 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:45:24.392062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:24.392062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:45:24.392062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:45:24.392062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:45:24.392062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:45:24.392062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:45:24.392062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:45:24.392062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:45:24.392062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:45:24.392062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:45:24.392062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:45:24.392062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:24.392062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:45:24.392062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:45:24.392062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:45:24.392062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:45:24.393403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:45:24.393403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:45:24.393403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:45:24.393403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:45:24.393403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:45:24.393403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:45:24.393403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:45:24.393403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:45:24.393403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:45:24.393403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:45:24.393403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:45:24.393403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:45:24.393403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:45:24.393403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:45:24.393403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:45:24.393403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:45:24.394640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:45:24.394640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:45:24.394640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:45:24.394640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:45:24.394640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:45:24.394640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:45:24.394640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:45:24.394640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:45:24.394640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:45:24.394640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:45:24.394640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:45:24.394640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:45:24.394640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:45:24.394640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:45:24.394640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 654/1298] 10.222.0.78 () {56 vars in 1117 bytes} [Wed May 25 14:45:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:45:24.399362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc072744-d5b7-40db-99b7-f829effe34bf None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:24.417748 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:24.503123 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc072744-d5b7-40db-99b7-f829effe34bf None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:24.539259 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fb92c-6b55-422f-80b6-0af38157737f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:24.539923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fb92c-6b55-422f-80b6-0af38157737f None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:24.540197 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fb92c-6b55-422f-80b6-0af38157737f None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:24.540465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fb92c-6b55-422f-80b6-0af38157737f None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:24.541169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053fb92c-6b55-422f-80b6-0af38157737f None admin] RBAC: Adding query filter params (name=placement) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:24.541476 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053fb92c-6b55-422f-80b6-0af38157737f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:24.542878 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053fb92c-6b55-422f-80b6-0af38157737f None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:24.548879 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 645/1299] 10.222.0.78 () {56 vars in 1116 bytes} [Wed May 25 14:45:24 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:24.555433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd07a23-9eef-4614-95dd-6ba28597ff77 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:24.592055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:24.660387 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd07a23-9eef-4614-95dd-6ba28597ff77 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:24.692637 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09a39d-59d0-4a80-bcb6-14ad925f133a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:24.693286 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09a39d-59d0-4a80-bcb6-14ad925f133a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:24.693572 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09a39d-59d0-4a80-bcb6-14ad925f133a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:24.693847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09a39d-59d0-4a80-bcb6-14ad925f133a None admin] PATH_INFO: `/v3/endpoints` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:24.694680 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09a39d-59d0-4a80-bcb6-14ad925f133a None admin] RBAC: Adding query filter params (interface=public, service_id=ba4f2292762b4f42b38657b8d574f550, region_id=RegionOne) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:24.694994 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09a39d-59d0-4a80-bcb6-14ad925f133a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:24.696369 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09a39d-59d0-4a80-bcb6-14ad925f133a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:24.701294 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 655/1300] 10.222.0.78 () {56 vars in 1251 bytes} [Wed May 25 14:45:24 2022] GET /identity/v3/endpoints?service_id=ba4f2292762b4f42b38657b8d574f550&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:24.707705 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fb92c-6b55-422f-80b6-0af38157737f None admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:24.788083 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:24.805774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fb92c-6b55-422f-80b6-0af38157737f None admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:24.843237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef70c5-10bc-4c02-8b58-a12b6592c910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:24.844189 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef70c5-10bc-4c02-8b58-a12b6592c910 None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:24.844607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef70c5-10bc-4c02-8b58-a12b6592c910 None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:24.845013 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef70c5-10bc-4c02-8b58-a12b6592c910 None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:24.846108 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef70c5-10bc-4c02-8b58-a12b6592c910 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:24.848234 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef70c5-10bc-4c02-8b58-a12b6592c910 None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:24.857827 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 646/1301] 10.222.0.78 () {56 vars in 1087 bytes} [Wed May 25 14:45:24 2022] GET /identity/v3/services => generated 2191 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:45:27.044207 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b23fc-cb37-42eb-b5df-5da4099cb785 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:27.044800 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b23fc-cb37-42eb-b5df-5da4099cb785 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:27.044800 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b23fc-cb37-42eb-b5df-5da4099cb785 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:27.046159 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 656/1302] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:45:27.051708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46665d87-908a-42cf-b0e8-6542cdaee134 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:27.052151 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46665d87-908a-42cf-b0e8-6542cdaee134 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:27.052260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46665d87-908a-42cf-b0e8-6542cdaee134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:27.070261 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-46665d87-908a-42cf-b0e8-6542cdaee134 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:27.071469 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-46665d87-908a-42cf-b0e8-6542cdaee134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:27.144112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 647/1303] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:45:27 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:45:27.168004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b23fc-cb37-42eb-b5df-5da4099cb785 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:27.284089 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:27.293713 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b23fc-cb37-42eb-b5df-5da4099cb785 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:27.329254 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75767b-59e3-47ff-a9a2-b93e2dc87e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:27.330826 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75767b-59e3-47ff-a9a2-b93e2dc87e6e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:27.331113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75767b-59e3-47ff-a9a2-b93e2dc87e6e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:27.331387 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75767b-59e3-47ff-a9a2-b93e2dc87e6e None admin] PATH_INFO: `/v3/services/placement` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:27.332036 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab75767b-59e3-47ff-a9a2-b93e2dc87e6e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:27.333416 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab75767b-59e3-47ff-a9a2-b93e2dc87e6e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:27.345292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-ab75767b-59e3-47ff-a9a2-b93e2dc87e6e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:45:27.345292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:27.345292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:45:27.345292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:45:27.345292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:45:27.345292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:45:27.345292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:45:27.345292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:45:27.345292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:45:27.345292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:45:27.345292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:45:27.345292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:45:27.345292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:45:27.345292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:45:27.345292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:45:27.345292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:45:27.345292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:45:27.347191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:45:27.347191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:45:27.347191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:45:27.347191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:45:27.347191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:45:27.347191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:45:27.347191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:45:27.347191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:45:27.347191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:45:27.347191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:45:27.347191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:45:27.347191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:45:27.347191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:45:27.347191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:45:27.347191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:45:27.347191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:45:27.348353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:45:27.348353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( May 25 14:45:27.348353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:45:27.348353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() May 25 14:45:27.348353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:45:27.348353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:45:27.348353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:45:27.348353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() May 25 14:45:27.348353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:45:27.348353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( May 25 14:45:27.348353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:45:27.348353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:45:27.348353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:45:27.348353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application May 25 14:45:27.348353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 657/1304] 10.222.0.78 () {56 vars in 1117 bytes} [Wed May 25 14:45:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:45:27.353458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46665d87-908a-42cf-b0e8-6542cdaee134 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:27.430661 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:27.453719 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46665d87-908a-42cf-b0e8-6542cdaee134 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:27.489026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3875964-eb3b-49d0-945e-39d6aaa147cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:27.489684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3875964-eb3b-49d0-945e-39d6aaa147cc None admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:27.489960 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3875964-eb3b-49d0-945e-39d6aaa147cc None admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:27.490258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3875964-eb3b-49d0-945e-39d6aaa147cc None admin] PATH_INFO: `/v3/services` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:27.490959 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3875964-eb3b-49d0-945e-39d6aaa147cc None admin] RBAC: Adding query filter params (name=placement) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:27.491257 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3875964-eb3b-49d0-945e-39d6aaa147cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:27.492621 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3875964-eb3b-49d0-945e-39d6aaa147cc None admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:27.498251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 648/1305] 10.222.0.78 () {56 vars in 1116 bytes} [Wed May 25 14:45:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:45:27.505598 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75767b-59e3-47ff-a9a2-b93e2dc87e6e None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:27.609795 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75767b-59e3-47ff-a9a2-b93e2dc87e6e None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:27.643591 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544dbc31-c961-494b-9ed9-2a9e768faa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:27.644268 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544dbc31-c961-494b-9ed9-2a9e768faa13 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:27.644554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544dbc31-c961-494b-9ed9-2a9e768faa13 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:27.644829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544dbc31-c961-494b-9ed9-2a9e768faa13 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:27.645558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544dbc31-c961-494b-9ed9-2a9e768faa13 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:27.646971 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544dbc31-c961-494b-9ed9-2a9e768faa13 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:27.665262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 658/1306] 10.222.0.78 () {60 vars in 1144 bytes} [Wed May 25 14:45:27 2022] POST /identity/v3/endpoints => generated 331 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:45:37.495038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=30 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:37.660963 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:39.380228 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37fd83-5486-46b7-89e0-848319744b42 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:39.381026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37fd83-5486-46b7-89e0-848319744b42 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:39.381026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37fd83-5486-46b7-89e0-848319744b42 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:39.382456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 649/1307] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:39.388052 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414e55c-c7e2-49a3-8190-2436de0bd956 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:39.388780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414e55c-c7e2-49a3-8190-2436de0bd956 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:39.388780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414e55c-c7e2-49a3-8190-2436de0bd956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:39.476503 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-d414e55c-c7e2-49a3-8190-2436de0bd956 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:39.477735 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d414e55c-c7e2-49a3-8190-2436de0bd956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:39.622655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 659/1308] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:46:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:46:39.675163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c3fae-5625-42c4-891a-c22b8f3e2ea3 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:39.675637 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c3fae-5625-42c4-891a-c22b8f3e2ea3 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:39.676039 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c3fae-5625-42c4-891a-c22b8f3e2ea3 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:39.677948 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 650/1309] 10.222.0.78 () {54 vars in 964 bytes} [Wed May 25 14:46:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:39.685740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967b433-e087-49a1-8b7a-b968456d432c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:39.686224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967b433-e087-49a1-8b7a-b968456d432c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:39.686666 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967b433-e087-49a1-8b7a-b968456d432c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:39.820892 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-f967b433-e087-49a1-8b7a-b968456d432c None None] MFA Rules not processed for user `70ad51a8ad68440da6618891e2e08cb8`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:39.822886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f967b433-e087-49a1-8b7a-b968456d432c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:39.864557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:39.981949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 660/1310] 10.222.0.78 () {58 vars in 1061 bytes} [Wed May 25 14:46:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:39.998307 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c3fae-5625-42c4-891a-c22b8f3e2ea3 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:40.000923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-bb8c3fae-5625-42c4-891a-c22b8f3e2ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:40.152386 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c3fae-5625-42c4-891a-c22b8f3e2ea3 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:40.246667 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf51ce-ed49-453b-9532-fb4e29490138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:40.247364 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf51ce-ed49-453b-9532-fb4e29490138 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:40.247642 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf51ce-ed49-453b-9532-fb4e29490138 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:40.247919 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf51ce-ed49-453b-9532-fb4e29490138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:40.249958 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-71bf51ce-ed49-453b-9532-fb4e29490138 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:40.325851 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:40.408935 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf51ce-ed49-453b-9532-fb4e29490138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:40.411028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf51ce-ed49-453b-9532-fb4e29490138 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:40.435097 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 651/1311] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:46:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:46:43.666033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a376fa-b3dc-43fd-81e7-4d5151c52408 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:43.666725 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a376fa-b3dc-43fd-81e7-4d5151c52408 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:43.666725 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a376fa-b3dc-43fd-81e7-4d5151c52408 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:43.668266 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 661/1312] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:43.673743 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1bfa4-a9a2-4d9c-8074-b1765de8bec0 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:43.674293 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1bfa4-a9a2-4d9c-8074-b1765de8bec0 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:43.674384 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1bfa4-a9a2-4d9c-8074-b1765de8bec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:43.777506 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-cee1bfa4-a9a2-4d9c-8074-b1765de8bec0 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:43.778802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-cee1bfa4-a9a2-4d9c-8074-b1765de8bec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:43.932367 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 652/1313] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:43.969955 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a376fa-b3dc-43fd-81e7-4d5151c52408 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:43.971553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d4a376fa-b3dc-43fd-81e7-4d5151c52408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:44.101630 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a376fa-b3dc-43fd-81e7-4d5151c52408 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:44.193417 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:44.199307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74047e0-9f7f-498b-9568-26371379938e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:44.199984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74047e0-9f7f-498b-9568-26371379938e service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:44.200270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74047e0-9f7f-498b-9568-26371379938e service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:44.200551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74047e0-9f7f-498b-9568-26371379938e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:44.202342 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b74047e0-9f7f-498b-9568-26371379938e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:44.362075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74047e0-9f7f-498b-9568-26371379938e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:44.364128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74047e0-9f7f-498b-9568-26371379938e service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:44.390767 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 662/1314] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:46:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:46:47.188457 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f6054-8848-4171-852e-6eb6853aebe6 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:47.188991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f6054-8848-4171-852e-6eb6853aebe6 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:47.188991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f6054-8848-4171-852e-6eb6853aebe6 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:47.190380 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 653/1315] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:47.196993 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557dc833-a866-400e-82e0-87fe7ba52332 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:47.197512 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557dc833-a866-400e-82e0-87fe7ba52332 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:47.197600 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557dc833-a866-400e-82e0-87fe7ba52332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:47.274393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-557dc833-a866-400e-82e0-87fe7ba52332 None None] MFA Rules not processed for user `74e87dd6b03d4b6ca41070df6e3f2f11`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:47.275687 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-557dc833-a866-400e-82e0-87fe7ba52332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:47.334790 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:47.383698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 663/1316] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:46:47 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:47.440592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa85e5-c196-4d5b-90f4-7ff5572b2260 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:47.440804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa85e5-c196-4d5b-90f4-7ff5572b2260 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:47.441042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa85e5-c196-4d5b-90f4-7ff5572b2260 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:47.443065 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 654/1317] 10.222.0.78 () {54 vars in 964 bytes} [Wed May 25 14:46:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:47.450776 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f593d5-7fa5-4d5b-89c7-23c0bf58990d None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:47.451063 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f593d5-7fa5-4d5b-89c7-23c0bf58990d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:47.451346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f593d5-7fa5-4d5b-89c7-23c0bf58990d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:47.543311 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-70f593d5-7fa5-4d5b-89c7-23c0bf58990d None None] MFA Rules not processed for user `70ad51a8ad68440da6618891e2e08cb8`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:47.544572 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-70f593d5-7fa5-4d5b-89c7-23c0bf58990d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:47.633083 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 664/1318] 10.222.0.78 () {58 vars in 1061 bytes} [Wed May 25 14:46:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:46:47.643096 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa85e5-c196-4d5b-90f4-7ff5572b2260 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:47.645639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-42fa85e5-c196-4d5b-90f4-7ff5572b2260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:47.656749 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:47.767181 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa85e5-c196-4d5b-90f4-7ff5572b2260 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:47.854815 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dea3a-ab2d-4a2e-8a15-adc0ceef7433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:47.855474 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dea3a-ab2d-4a2e-8a15-adc0ceef7433 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:47.855751 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dea3a-ab2d-4a2e-8a15-adc0ceef7433 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:47.856024 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dea3a-ab2d-4a2e-8a15-adc0ceef7433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:47.857750 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f27dea3a-ab2d-4a2e-8a15-adc0ceef7433 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:47.933178 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:47.989184 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27dea3a-ab2d-4a2e-8a15-adc0ceef7433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:47.990744 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27dea3a-ab2d-4a2e-8a15-adc0ceef7433 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:48.013086 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 655/1319] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:46:47 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:46:51.330173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beef2cd-0e65-4777-bb0f-87a5ca135cb9 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:51.330781 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beef2cd-0e65-4777-bb0f-87a5ca135cb9 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:51.330781 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beef2cd-0e65-4777-bb0f-87a5ca135cb9 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:51.332678 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 665/1320] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:46:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:51.338802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8b7d9-4b79-46ab-8dbf-b3f85b05f44d None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:51.339305 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8b7d9-4b79-46ab-8dbf-b3f85b05f44d None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:51.339442 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8b7d9-4b79-46ab-8dbf-b3f85b05f44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:51.406359 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-23a8b7d9-4b79-46ab-8dbf-b3f85b05f44d None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:51.407582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-23a8b7d9-4b79-46ab-8dbf-b3f85b05f44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:51.506384 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 656/1321] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:46:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:51.545526 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beef2cd-0e65-4777-bb0f-87a5ca135cb9 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:51.547160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-6beef2cd-0e65-4777-bb0f-87a5ca135cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:51.607296 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:51.669277 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beef2cd-0e65-4777-bb0f-87a5ca135cb9 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:51.777054 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a883a-37cc-4553-9633-8608f9f73ccc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:51.778263 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a883a-37cc-4553-9633-8608f9f73ccc service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:51.778765 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a883a-37cc-4553-9633-8608f9f73ccc service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:51.779262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a883a-37cc-4553-9633-8608f9f73ccc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:51.782648 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9d8a883a-37cc-4553-9633-8608f9f73ccc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:51.923122 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:51.936898 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a883a-37cc-4553-9633-8608f9f73ccc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:51.938561 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a883a-37cc-4553-9633-8608f9f73ccc service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:51.959515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 666/1322] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:46:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:46:55.003469 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34cf7de-1249-421c-bbeb-927846fcc383 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:55.004380 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34cf7de-1249-421c-bbeb-927846fcc383 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:55.004380 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34cf7de-1249-421c-bbeb-927846fcc383 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:55.005503 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 657/1323] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:55.011476 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74440c-8b90-4111-a578-50ba9bb2552c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:55.012230 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74440c-8b90-4111-a578-50ba9bb2552c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:55.012230 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74440c-8b90-4111-a578-50ba9bb2552c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:55.118279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-7b74440c-8b90-4111-a578-50ba9bb2552c None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:55.120283 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7b74440c-8b90-4111-a578-50ba9bb2552c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:55.262801 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 667/1324] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:46:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:46:55.300256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34cf7de-1249-421c-bbeb-927846fcc383 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:55.301858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d34cf7de-1249-421c-bbeb-927846fcc383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:55.321936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:55.441274 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34cf7de-1249-421c-bbeb-927846fcc383 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:55.578200 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd7587-ce04-4e02-aa06-d78525313810 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:55.579359 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd7587-ce04-4e02-aa06-d78525313810 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:55.579860 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd7587-ce04-4e02-aa06-d78525313810 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:55.580359 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd7587-ce04-4e02-aa06-d78525313810 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:55.583702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-74fd7587-ce04-4e02-aa06-d78525313810 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:55.711392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:55.773172 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd7587-ce04-4e02-aa06-d78525313810 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:55.774677 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd7587-ce04-4e02-aa06-d78525313810 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:55.799542 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 658/1325] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:46:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:46:58.174205 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7edbc-247c-4403-93f7-11d3adef5373 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:58.174747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7edbc-247c-4403-93f7-11d3adef5373 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:58.174747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7edbc-247c-4403-93f7-11d3adef5373 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:58.176105 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 668/1326] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:46:58.183860 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e117235-4e69-45bd-abe6-71bb835ecd9c None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:58.188394 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e117235-4e69-45bd-abe6-71bb835ecd9c None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:58.188394 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e117235-4e69-45bd-abe6-71bb835ecd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:58.280874 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-9e117235-4e69-45bd-abe6-71bb835ecd9c None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:58.283034 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-9e117235-4e69-45bd-abe6-71bb835ecd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:58.391657 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 659/1327] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:46:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:58.431406 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7edbc-247c-4403-93f7-11d3adef5373 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:58.433514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d6c7edbc-247c-4403-93f7-11d3adef5373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:58.531607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:58.584386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7edbc-247c-4403-93f7-11d3adef5373 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:58.678059 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa82dc-824b-40bd-bdcf-9ac00cb7b11a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:58.678700 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa82dc-824b-40bd-bdcf-9ac00cb7b11a service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:58.678970 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa82dc-824b-40bd-bdcf-9ac00cb7b11a service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:58.679244 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa82dc-824b-40bd-bdcf-9ac00cb7b11a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:58.680898 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-54aa82dc-824b-40bd-bdcf-9ac00cb7b11a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:58.806205 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:58.809477 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa82dc-824b-40bd-bdcf-9ac00cb7b11a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:58.811309 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa82dc-824b-40bd-bdcf-9ac00cb7b11a service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:58.836901 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 669/1328] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:46:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:47:01.193261 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725be4a-6cb2-4ceb-8eda-e915ebece970 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:01.193775 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725be4a-6cb2-4ceb-8eda-e915ebece970 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:01.193775 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725be4a-6cb2-4ceb-8eda-e915ebece970 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:01.195435 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 660/1329] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:47:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:01.201255 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548915e2-a37d-47f6-8a31-18b26c467cd2 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:01.201793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548915e2-a37d-47f6-8a31-18b26c467cd2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:01.201915 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548915e2-a37d-47f6-8a31-18b26c467cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:01.271654 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-548915e2-a37d-47f6-8a31-18b26c467cd2 None None] MFA Rules not processed for user `74e87dd6b03d4b6ca41070df6e3f2f11`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:01.272876 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-548915e2-a37d-47f6-8a31-18b26c467cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:01.375194 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 670/1330] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:47:01 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:01.412638 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725be4a-6cb2-4ceb-8eda-e915ebece970 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:01.414332 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-3725be4a-6cb2-4ceb-8eda-e915ebece970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:01.446291 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:47:01.520868 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725be4a-6cb2-4ceb-8eda-e915ebece970 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:01.599136 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38bb75e-8630-4dd6-a033-e64d92d7376b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:01.599882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bb75e-8630-4dd6-a033-e64d92d7376b service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:01.600164 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bb75e-8630-4dd6-a033-e64d92d7376b service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:01.600441 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bb75e-8630-4dd6-a033-e64d92d7376b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:01.602188 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f38bb75e-8630-4dd6-a033-e64d92d7376b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:01.684284 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:47:01.721317 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38bb75e-8630-4dd6-a033-e64d92d7376b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:01.722748 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38bb75e-8630-4dd6-a033-e64d92d7376b service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:01.743162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 661/1331] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:47:01 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:04.279386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04674d5b-0bb3-4e16-83b1-7db444d64a54 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:04.280187 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04674d5b-0bb3-4e16-83b1-7db444d64a54 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:04.280187 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04674d5b-0bb3-4e16-83b1-7db444d64a54 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:04.281560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 671/1332] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:04.288059 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a19e9-5932-40a8-8bba-2cf6bffe7a5c None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:04.288625 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a19e9-5932-40a8-8bba-2cf6bffe7a5c None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:04.288625 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a19e9-5932-40a8-8bba-2cf6bffe7a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:04.364554 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-fe6a19e9-5932-40a8-8bba-2cf6bffe7a5c None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:04.365770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-fe6a19e9-5932-40a8-8bba-2cf6bffe7a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:04.468161 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 662/1333] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:47:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:04.512594 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04674d5b-0bb3-4e16-83b1-7db444d64a54 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:04.514207 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-04674d5b-0bb3-4e16-83b1-7db444d64a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:04.597155 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:47:04.636747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04674d5b-0bb3-4e16-83b1-7db444d64a54 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:04.717626 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f1bc9-1191-448a-b754-bd5127090f64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:04.718315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f1bc9-1191-448a-b754-bd5127090f64 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:04.718602 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f1bc9-1191-448a-b754-bd5127090f64 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:04.718876 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f1bc9-1191-448a-b754-bd5127090f64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:04.720555 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-007f1bc9-1191-448a-b754-bd5127090f64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:04.851908 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f1bc9-1191-448a-b754-bd5127090f64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:04.853470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f1bc9-1191-448a-b754-bd5127090f64 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:04.865378 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:47:04.877443 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 672/1334] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:47:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:07.216000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241836a-1eed-4b63-9226-54df688e7d66 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:07.216577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241836a-1eed-4b63-9226-54df688e7d66 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:07.216577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241836a-1eed-4b63-9226-54df688e7d66 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:07.217997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 663/1335] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:07.224535 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e6cfa-2dab-41f0-9302-2246b9dc6ecf None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:07.225052 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e6cfa-2dab-41f0-9302-2246b9dc6ecf None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:07.225218 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e6cfa-2dab-41f0-9302-2246b9dc6ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:07.293942 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-706e6cfa-2dab-41f0-9302-2246b9dc6ecf None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:07.295213 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-706e6cfa-2dab-41f0-9302-2246b9dc6ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:07.401304 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 673/1336] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:07.443434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241836a-1eed-4b63-9226-54df688e7d66 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:07.445902 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-7241836a-1eed-4b63-9226-54df688e7d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:07.490142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:47:07.561229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241836a-1eed-4b63-9226-54df688e7d66 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:07.636867 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc0381-d313-41ce-a60a-18a9f5542af9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:07.637507 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc0381-d313-41ce-a60a-18a9f5542af9 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:07.637782 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc0381-d313-41ce-a60a-18a9f5542af9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:07.638073 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc0381-d313-41ce-a60a-18a9f5542af9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:07.639710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-57bc0381-d313-41ce-a60a-18a9f5542af9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:07.725503 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:47:07.752644 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc0381-d313-41ce-a60a-18a9f5542af9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:07.754719 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc0381-d313-41ce-a60a-18a9f5542af9 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:07.773684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 664/1337] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:47:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:10.439112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfbff9-4c2c-4d80-a61b-69a04259b137 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:10.439654 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfbff9-4c2c-4d80-a61b-69a04259b137 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:10.439654 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfbff9-4c2c-4d80-a61b-69a04259b137 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:10.441561 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 674/1338] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:10.447043 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca017989-1b22-45d2-bccf-7e1743acdf3d None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:10.447517 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca017989-1b22-45d2-bccf-7e1743acdf3d None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:10.447621 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca017989-1b22-45d2-bccf-7e1743acdf3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:10.516619 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-ca017989-1b22-45d2-bccf-7e1743acdf3d None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:10.517811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-ca017989-1b22-45d2-bccf-7e1743acdf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:10.639925 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 665/1339] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:47:10 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:10.677590 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfbff9-4c2c-4d80-a61b-69a04259b137 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:10.680025 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-49cfbff9-4c2c-4d80-a61b-69a04259b137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:10.764325 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:47:10.780980 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfbff9-4c2c-4d80-a61b-69a04259b137 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:10.784621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca26ba-7e9f-4a8d-89df-a59fc3ed930b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:10.785897 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca26ba-7e9f-4a8d-89df-a59fc3ed930b service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:10.786217 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca26ba-7e9f-4a8d-89df-a59fc3ed930b service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:10.786491 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca26ba-7e9f-4a8d-89df-a59fc3ed930b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:10.795978 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca26ba-7e9f-4a8d-89df-a59fc3ed930b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:10.797794 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca26ba-7e9f-4a8d-89df-a59fc3ed930b service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:10.807810 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 675/1340] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:47:10 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:47:15.451812 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51e556-7c17-4aef-b3fa-5a2cfa2bdec9 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:15.452389 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51e556-7c17-4aef-b3fa-5a2cfa2bdec9 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:15.452591 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51e556-7c17-4aef-b3fa-5a2cfa2bdec9 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:15.454133 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 666/1341] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:47:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:47:15.460027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e1b55-fd76-4a66-89d0-cebc77a109df None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:15.460528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e1b55-fd76-4a66-89d0-cebc77a109df None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:15.460689 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e1b55-fd76-4a66-89d0-cebc77a109df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:15.490882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-869e1b55-fd76-4a66-89d0-cebc77a109df None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:15.492080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-869e1b55-fd76-4a66-89d0-cebc77a109df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:15.575279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 676/1342] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:47:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:15.627146 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51e556-7c17-4aef-b3fa-5a2cfa2bdec9 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:15.639016 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51e556-7c17-4aef-b3fa-5a2cfa2bdec9 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:15.642596 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d4a74-f58f-4e7a-b120-40bfcede9fe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:15.643256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d4a74-f58f-4e7a-b120-40bfcede9fe1 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:15.643535 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d4a74-f58f-4e7a-b120-40bfcede9fe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:15.643805 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d4a74-f58f-4e7a-b120-40bfcede9fe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:15.654429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d4a74-f58f-4e7a-b120-40bfcede9fe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:15.656147 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d4a74-f58f-4e7a-b120-40bfcede9fe1 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:15.667136 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 667/1343] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:47:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:18.628094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c515ed-84f6-49a4-b1d3-5604a34693e8 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:18.628651 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c515ed-84f6-49a4-b1d3-5604a34693e8 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:18.628773 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c515ed-84f6-49a4-b1d3-5604a34693e8 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:18.630174 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 677/1344] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:18.636268 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3f078-c2bd-459c-8271-e747592fb378 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:18.636773 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3f078-c2bd-459c-8271-e747592fb378 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:18.636885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3f078-c2bd-459c-8271-e747592fb378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:18.655981 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-69c3f078-c2bd-459c-8271-e747592fb378 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:18.657211 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-69c3f078-c2bd-459c-8271-e747592fb378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:18.666041 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 668/1345] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:47:18 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:18.703404 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c515ed-84f6-49a4-b1d3-5604a34693e8 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:18.714620 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c515ed-84f6-49a4-b1d3-5604a34693e8 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:18.718235 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc20a5c-e8f2-4eff-ac4b-d6030398a60b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:18.718887 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc20a5c-e8f2-4eff-ac4b-d6030398a60b service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:18.719173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc20a5c-e8f2-4eff-ac4b-d6030398a60b service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:18.719444 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc20a5c-e8f2-4eff-ac4b-d6030398a60b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:18.729958 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc20a5c-e8f2-4eff-ac4b-d6030398a60b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:18.731441 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc20a5c-e8f2-4eff-ac4b-d6030398a60b service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:18.742389 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 678/1346] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:47:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:23.305281 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a6396-d99a-44d5-b078-05c618135cd3 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:23.305833 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a6396-d99a-44d5-b078-05c618135cd3 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:23.305833 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a6396-d99a-44d5-b078-05c618135cd3 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:23.307369 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 669/1347] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:23.312771 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75676984-51ea-48a7-963b-6c85a10557e5 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:23.313200 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75676984-51ea-48a7-963b-6c85a10557e5 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:23.313314 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75676984-51ea-48a7-963b-6c85a10557e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:23.332422 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-75676984-51ea-48a7-963b-6c85a10557e5 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:23.333627 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-75676984-51ea-48a7-963b-6c85a10557e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:23.341773 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 679/1348] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:47:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:23.380173 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a6396-d99a-44d5-b078-05c618135cd3 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:23.391832 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a6396-d99a-44d5-b078-05c618135cd3 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:23.395751 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e07c0b-a33e-4127-a571-e03c0294a179 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:23.396434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e07c0b-a33e-4127-a571-e03c0294a179 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:23.396718 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e07c0b-a33e-4127-a571-e03c0294a179 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:23.396991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e07c0b-a33e-4127-a571-e03c0294a179 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:23.407536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e07c0b-a33e-4127-a571-e03c0294a179 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:23.408998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e07c0b-a33e-4127-a571-e03c0294a179 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:23.420345 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 670/1349] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:47:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:29.498684 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d934e0-bab2-4289-a5f0-88599b1a6e62 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:29.499270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d934e0-bab2-4289-a5f0-88599b1a6e62 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:29.499895 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d934e0-bab2-4289-a5f0-88599b1a6e62 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:29.502287 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 680/1350] 10.222.0.78 () {54 vars in 921 bytes} [Wed May 25 14:47:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:29.522585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e8db0-25db-4a39-b72f-00647d51f3f3 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:29.523050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e8db0-25db-4a39-b72f-00647d51f3f3 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:29.523191 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e8db0-25db-4a39-b72f-00647d51f3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:29.539860 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:47:29.606921 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-4d4e8db0-25db-4a39-b72f-00647d51f3f3 None None] MFA Rules not processed for user `63789c0a113447ff90933d447cf5bac3`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:29.608407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-4d4e8db0-25db-4a39-b72f-00647d51f3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:29.704228 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 671/1351] 10.222.0.78 () {58 vars in 1018 bytes} [Wed May 25 14:47:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:33.327801 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=57 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:47:34.382325 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f739c27-a0c2-4dac-9f97-2e57faa98773 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:34.383273 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f739c27-a0c2-4dac-9f97-2e57faa98773 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:34.383273 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f739c27-a0c2-4dac-9f97-2e57faa98773 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:34.384913 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 681/1352] 10.222.0.78 () {54 vars in 921 bytes} [Wed May 25 14:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:47:34.390789 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d9700-2142-47f6-8912-49f747dedb41 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:34.391352 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d9700-2142-47f6-8912-49f747dedb41 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:34.391786 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d9700-2142-47f6-8912-49f747dedb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:34.413619 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-868d9700-2142-47f6-8912-49f747dedb41 None None] MFA Rules not processed for user `63789c0a113447ff90933d447cf5bac3`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:34.415201 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-868d9700-2142-47f6-8912-49f747dedb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:34.424813 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 672/1353] 10.222.0.78 () {58 vars in 1018 bytes} [Wed May 25 14:47:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:36.418301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483abfc-7229-45c4-ba5d-26247c8746af None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:36.422574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483abfc-7229-45c4-ba5d-26247c8746af None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:36.422574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483abfc-7229-45c4-ba5d-26247c8746af None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:36.422574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 682/1354] 10.222.0.78 () {54 vars in 921 bytes} [Wed May 25 14:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:36.426085 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e48a4-68dd-495e-b49f-b83d8d7cda50 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:36.426517 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e48a4-68dd-495e-b49f-b83d8d7cda50 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:36.426629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e48a4-68dd-495e-b49f-b83d8d7cda50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:36.446584 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-f85e48a4-68dd-495e-b49f-b83d8d7cda50 None None] MFA Rules not processed for user `63789c0a113447ff90933d447cf5bac3`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:36.447788 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f85e48a4-68dd-495e-b49f-b83d8d7cda50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:36.458526 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 673/1355] 10.222.0.78 () {58 vars in 1018 bytes} [Wed May 25 14:47:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:40.573581 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df2c0e-cad5-4187-81fd-d0d20dd0e518 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:40.573581 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df2c0e-cad5-4187-81fd-d0d20dd0e518 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:40.573581 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df2c0e-cad5-4187-81fd-d0d20dd0e518 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:40.576927 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 683/1356] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:47:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:40.582636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3564be-9f8c-4ecb-b64f-10b17af2a862 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:40.583527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3564be-9f8c-4ecb-b64f-10b17af2a862 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:40.584522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3564be-9f8c-4ecb-b64f-10b17af2a862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:40.615443 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-dc3564be-9f8c-4ecb-b64f-10b17af2a862 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:40.617436 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-dc3564be-9f8c-4ecb-b64f-10b17af2a862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:40.633408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 674/1357] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:47:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:40.704374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b50883-9b5c-4e00-b7d9-41ee7d17d9e2 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:40.705458 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b50883-9b5c-4e00-b7d9-41ee7d17d9e2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:40.706396 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b50883-9b5c-4e00-b7d9-41ee7d17d9e2 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:40.710599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 684/1358] 10.222.0.78 () {54 vars in 961 bytes} [Wed May 25 14:47:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:40.730337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbed993-1ece-4f21-ac4b-4a31d9aafd35 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:40.731346 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbed993-1ece-4f21-ac4b-4a31d9aafd35 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:40.732292 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbed993-1ece-4f21-ac4b-4a31d9aafd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:40.810456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-cfbed993-1ece-4f21-ac4b-4a31d9aafd35 None None] MFA Rules not processed for user `e34e1d7ffbac49b0bde45829af0c6925`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:40.811692 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-cfbed993-1ece-4f21-ac4b-4a31d9aafd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:40.893976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:47:40.922934 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 675/1359] 10.222.0.78 () {58 vars in 1058 bytes} [Wed May 25 14:47:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:40.931906 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b50883-9b5c-4e00-b7d9-41ee7d17d9e2 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:40.948325 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b50883-9b5c-4e00-b7d9-41ee7d17d9e2 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:40.954268 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32079506-ffc4-4b4c-8667-5994b5b612be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:40.955089 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32079506-ffc4-4b4c-8667-5994b5b612be service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:40.955375 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32079506-ffc4-4b4c-8667-5994b5b612be service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:40.955652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32079506-ffc4-4b4c-8667-5994b5b612be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:40.965785 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32079506-ffc4-4b4c-8667-5994b5b612be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:40.967526 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32079506-ffc4-4b4c-8667-5994b5b612be service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:40.980549 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 685/1360] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:47:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:43.548725 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b38e5-7107-4cea-9490-c54958d4ff8d None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:43.549290 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b38e5-7107-4cea-9490-c54958d4ff8d None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:43.549290 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b38e5-7107-4cea-9490-c54958d4ff8d None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:43.552410 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 676/1361] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:47:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:43.558804 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774375ef-ed6b-408e-a31c-75b9f5baada9 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:43.559295 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774375ef-ed6b-408e-a31c-75b9f5baada9 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:43.559375 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774375ef-ed6b-408e-a31c-75b9f5baada9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:43.579957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-774375ef-ed6b-408e-a31c-75b9f5baada9 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:43.581229 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-774375ef-ed6b-408e-a31c-75b9f5baada9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:43.591072 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 686/1362] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:43.638126 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b38e5-7107-4cea-9490-c54958d4ff8d None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:43.656255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b38e5-7107-4cea-9490-c54958d4ff8d None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:43.662854 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8255678-c9dc-4a9c-9f5f-14b3a672eceb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:43.663503 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8255678-c9dc-4a9c-9f5f-14b3a672eceb service nova] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:43.663783 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8255678-c9dc-4a9c-9f5f-14b3a672eceb service nova] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:43.664053 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8255678-c9dc-4a9c-9f5f-14b3a672eceb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:43.673566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8255678-c9dc-4a9c-9f5f-14b3a672eceb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:43.675218 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8255678-c9dc-4a9c-9f5f-14b3a672eceb service nova] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:43.687353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 677/1363] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:47:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:46.307283 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52902743-04d5-4c9f-bd11-0930dc19079a None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:46.307846 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52902743-04d5-4c9f-bd11-0930dc19079a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:46.307846 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52902743-04d5-4c9f-bd11-0930dc19079a None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:46.309223 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 687/1364] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:46.315039 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c0c8a-3400-4cfa-ab5d-ebeeee2ab64e None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:46.315472 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c0c8a-3400-4cfa-ab5d-ebeeee2ab64e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:46.315617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c0c8a-3400-4cfa-ab5d-ebeeee2ab64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:46.334669 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-760c0c8a-3400-4cfa-ab5d-ebeeee2ab64e None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:46.335872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-760c0c8a-3400-4cfa-ab5d-ebeeee2ab64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:46.344845 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 678/1365] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:47:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:46.390683 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fc324-22de-49ab-b290-6a46097464f4 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:46.391050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fc324-22de-49ab-b290-6a46097464f4 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:46.391397 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fc324-22de-49ab-b290-6a46097464f4 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:46.392841 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 688/1366] 10.222.0.78 () {54 vars in 961 bytes} [Wed May 25 14:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:46.400182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa87ad-9874-4ac8-a6a6-2c4401495060 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:46.400559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa87ad-9874-4ac8-a6a6-2c4401495060 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:46.400997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa87ad-9874-4ac8-a6a6-2c4401495060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:46.421342 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-69fa87ad-9874-4ac8-a6a6-2c4401495060 None None] MFA Rules not processed for user `e34e1d7ffbac49b0bde45829af0c6925`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:46.422895 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-69fa87ad-9874-4ac8-a6a6-2c4401495060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:46.436380 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 679/1367] 10.222.0.78 () {58 vars in 1058 bytes} [Wed May 25 14:47:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:46.445120 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9fc324-22de-49ab-b290-6a46097464f4 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:46.461971 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9fc324-22de-49ab-b290-6a46097464f4 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:46.467522 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d7f30-91ec-4dab-853f-cdb5f5875717 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:46.468181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d7f30-91ec-4dab-853f-cdb5f5875717 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:46.468464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d7f30-91ec-4dab-853f-cdb5f5875717 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:46.468766 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d7f30-91ec-4dab-853f-cdb5f5875717 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:46.477825 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d7f30-91ec-4dab-853f-cdb5f5875717 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:46.479449 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d7f30-91ec-4dab-853f-cdb5f5875717 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:46.489012 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 689/1368] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:47:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:48.827720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115ac15-7ff4-4764-95ee-3a709da91252 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:48.828286 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115ac15-7ff4-4764-95ee-3a709da91252 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:48.828509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115ac15-7ff4-4764-95ee-3a709da91252 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:48.830146 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 680/1369] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:47:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:48.836110 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682517d4-cb8e-40f9-ae9d-3017a0bd27d5 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:48.836572 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682517d4-cb8e-40f9-ae9d-3017a0bd27d5 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:48.836671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682517d4-cb8e-40f9-ae9d-3017a0bd27d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:48.856415 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-682517d4-cb8e-40f9-ae9d-3017a0bd27d5 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:48.857770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-682517d4-cb8e-40f9-ae9d-3017a0bd27d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:48.867353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 690/1370] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:47:48.915355 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115ac15-7ff4-4764-95ee-3a709da91252 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:48.930031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115ac15-7ff4-4764-95ee-3a709da91252 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:48.935409 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e573955-0563-4d25-8ec1-52b36bf6432b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:48.936053 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e573955-0563-4d25-8ec1-52b36bf6432b service nova] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:48.936327 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e573955-0563-4d25-8ec1-52b36bf6432b service nova] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:48.936595 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e573955-0563-4d25-8ec1-52b36bf6432b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:48.946256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e573955-0563-4d25-8ec1-52b36bf6432b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:48.947871 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e573955-0563-4d25-8ec1-52b36bf6432b service nova] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:48.958592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 681/1371] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:47:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:47:49.644636 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824653f-ce3b-4d2b-8ab9-f9d55301bb29 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:49.644926 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824653f-ce3b-4d2b-8ab9-f9d55301bb29 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:49.645382 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824653f-ce3b-4d2b-8ab9-f9d55301bb29 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:49.647017 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 691/1372] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:47:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:49.653145 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0af37f-0bd9-47bc-9fb4-aafa76fe39a6 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:49.653442 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0af37f-0bd9-47bc-9fb4-aafa76fe39a6 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:49.653821 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0af37f-0bd9-47bc-9fb4-aafa76fe39a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:49.672545 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-1d0af37f-0bd9-47bc-9fb4-aafa76fe39a6 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:49.674271 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-1d0af37f-0bd9-47bc-9fb4-aafa76fe39a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:49.685715 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 682/1373] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:50.008049 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07db62-811a-49cb-a95c-11fa0e149783 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:50.008607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07db62-811a-49cb-a95c-11fa0e149783 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:50.008790 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07db62-811a-49cb-a95c-11fa0e149783 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:50.010588 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 692/1374] 10.222.0.78 () {54 vars in 963 bytes} [Wed May 25 14:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:50.016927 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02094489-0812-4da9-beec-67433e050f3c None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:50.017491 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02094489-0812-4da9-beec-67433e050f3c None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:50.017640 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02094489-0812-4da9-beec-67433e050f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:50.074763 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-02094489-0812-4da9-beec-67433e050f3c None None] MFA Rules not processed for user `50dbd3dd7e87481cb940a467c294006c`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:50.076083 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-02094489-0812-4da9-beec-67433e050f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:50.148097 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 683/1375] 10.222.0.78 () {58 vars in 1060 bytes} [Wed May 25 14:47:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:50.157331 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c07db62-811a-49cb-a95c-11fa0e149783 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:50.170270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c07db62-811a-49cb-a95c-11fa0e149783 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:50.174873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f6061-9e41-40eb-8b61-872d9235ddbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:50.176505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f6061-9e41-40eb-8b61-872d9235ddbf service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:50.176888 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f6061-9e41-40eb-8b61-872d9235ddbf service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:50.177264 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f6061-9e41-40eb-8b61-872d9235ddbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:50.187836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f6061-9e41-40eb-8b61-872d9235ddbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:50.189296 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f6061-9e41-40eb-8b61-872d9235ddbf service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:50.200475 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 693/1376] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:47:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:51.263053 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02fc38-251f-4aee-bac9-475570db1248 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:51.263606 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02fc38-251f-4aee-bac9-475570db1248 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:51.263606 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02fc38-251f-4aee-bac9-475570db1248 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:51.265172 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 684/1377] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:51.270832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae86b2-3dbf-4ebb-bd4d-51e7a52a8a8b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:51.271285 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae86b2-3dbf-4ebb-bd4d-51e7a52a8a8b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:51.271389 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae86b2-3dbf-4ebb-bd4d-51e7a52a8a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:51.289507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-f6ae86b2-3dbf-4ebb-bd4d-51e7a52a8a8b None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:51.290753 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f6ae86b2-3dbf-4ebb-bd4d-51e7a52a8a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:51.299944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 694/1378] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:47:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:51.345482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02fc38-251f-4aee-bac9-475570db1248 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:51.360207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02fc38-251f-4aee-bac9-475570db1248 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:51.366300 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f3a0e7-96f7-4882-a634-f8567a8cb8b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:51.366939 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3a0e7-96f7-4882-a634-f8567a8cb8b9 service nova] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:51.367219 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3a0e7-96f7-4882-a634-f8567a8cb8b9 service nova] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:51.367491 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3a0e7-96f7-4882-a634-f8567a8cb8b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:51.376543 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f3a0e7-96f7-4882-a634-f8567a8cb8b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:51.378170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f3a0e7-96f7-4882-a634-f8567a8cb8b9 service nova] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:51.389155 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 685/1379] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:47:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:53.710721 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c091e-850d-4611-ac4d-2348eff807b2 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:53.711252 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c091e-850d-4611-ac4d-2348eff807b2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:53.711252 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c091e-850d-4611-ac4d-2348eff807b2 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:53.712708 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 695/1380] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:53.717855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d18348-66de-46f5-9c47-3462d6d88ce8 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:53.718300 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d18348-66de-46f5-9c47-3462d6d88ce8 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:53.718446 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d18348-66de-46f5-9c47-3462d6d88ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:53.736715 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-12d18348-66de-46f5-9c47-3462d6d88ce8 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:53.738845 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-12d18348-66de-46f5-9c47-3462d6d88ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:53.747054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 686/1381] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:47:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:53.791456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c091e-850d-4611-ac4d-2348eff807b2 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:53.806877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c091e-850d-4611-ac4d-2348eff807b2 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:53.812418 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b92fe-88f6-471c-84eb-f8a0b43df8f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:53.813074 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b92fe-88f6-471c-84eb-f8a0b43df8f7 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:53.813358 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b92fe-88f6-471c-84eb-f8a0b43df8f7 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:53.813639 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b92fe-88f6-471c-84eb-f8a0b43df8f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:53.823130 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b92fe-88f6-471c-84eb-f8a0b43df8f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:53.824751 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b92fe-88f6-471c-84eb-f8a0b43df8f7 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:53.842259 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 696/1382] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:47:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:56.158385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f9863-f50a-4252-96f6-c8a813e422b8 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:56.158901 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f9863-f50a-4252-96f6-c8a813e422b8 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:56.158901 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f9863-f50a-4252-96f6-c8a813e422b8 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:56.165111 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 687/1383] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:47:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:56.167105 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b9903-6daa-4dca-892c-9ad961424871 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:56.168036 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b9903-6daa-4dca-892c-9ad961424871 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:56.168326 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b9903-6daa-4dca-892c-9ad961424871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:56.187519 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-7f2b9903-6daa-4dca-892c-9ad961424871 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:56.188724 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7f2b9903-6daa-4dca-892c-9ad961424871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:56.197258 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 697/1384] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:56.239413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f9863-f50a-4252-96f6-c8a813e422b8 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:56.254037 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f9863-f50a-4252-96f6-c8a813e422b8 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:56.259614 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5cfb2b-59c1-489f-8591-be870c527727 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:56.260497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cfb2b-59c1-489f-8591-be870c527727 service nova] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:56.260784 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cfb2b-59c1-489f-8591-be870c527727 service nova] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:56.261060 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cfb2b-59c1-489f-8591-be870c527727 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:56.270182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5cfb2b-59c1-489f-8591-be870c527727 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:56.271787 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5cfb2b-59c1-489f-8591-be870c527727 service nova] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:56.282763 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 688/1385] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:47:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:58.666587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7eb234-6f7c-4720-a433-ff526ef16495 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:58.667307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7eb234-6f7c-4720-a433-ff526ef16495 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:58.667307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7eb234-6f7c-4720-a433-ff526ef16495 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:58.668862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 698/1386] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:47:58.676057 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a77976-0610-4661-956a-7ab5fca32e99 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:58.676849 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a77976-0610-4661-956a-7ab5fca32e99 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:58.676849 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a77976-0610-4661-956a-7ab5fca32e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:58.695621 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-42a77976-0610-4661-956a-7ab5fca32e99 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:58.696852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-42a77976-0610-4661-956a-7ab5fca32e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:58.705137 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 689/1387] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:47:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:58.760239 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7eb234-6f7c-4720-a433-ff526ef16495 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:58.776201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7eb234-6f7c-4720-a433-ff526ef16495 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:58.783950 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c03a9-f223-4258-b32c-184059cdcb7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:58.784623 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c03a9-f223-4258-b32c-184059cdcb7a service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:58.784908 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c03a9-f223-4258-b32c-184059cdcb7a service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:58.785188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c03a9-f223-4258-b32c-184059cdcb7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:58.795975 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c03a9-f223-4258-b32c-184059cdcb7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:58.797676 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c03a9-f223-4258-b32c-184059cdcb7a service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:58.811797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 699/1388] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:47:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:48:01.322883 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54b231-0935-41f7-ae91-033adf0f0ad6 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:01.323445 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54b231-0935-41f7-ae91-033adf0f0ad6 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:01.323802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54b231-0935-41f7-ae91-033adf0f0ad6 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:01.326221 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 690/1389] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:48:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:01.332489 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cda594-38cf-4ddc-b02b-ca88f1f3fce9 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:01.333015 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cda594-38cf-4ddc-b02b-ca88f1f3fce9 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:01.333416 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cda594-38cf-4ddc-b02b-ca88f1f3fce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:01.356156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-d1cda594-38cf-4ddc-b02b-ca88f1f3fce9 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:01.357445 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d1cda594-38cf-4ddc-b02b-ca88f1f3fce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:01.366838 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 700/1390] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:48:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:48:01.419586 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54b231-0935-41f7-ae91-033adf0f0ad6 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:01.442513 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54b231-0935-41f7-ae91-033adf0f0ad6 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:01.451758 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee807f4-feae-4d6c-87f5-cdd3abb2d744 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:01.452934 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee807f4-feae-4d6c-87f5-cdd3abb2d744 service nova] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:01.453447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee807f4-feae-4d6c-87f5-cdd3abb2d744 service nova] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:01.453927 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee807f4-feae-4d6c-87f5-cdd3abb2d744 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:01.465155 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee807f4-feae-4d6c-87f5-cdd3abb2d744 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:01.466884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee807f4-feae-4d6c-87f5-cdd3abb2d744 service nova] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:01.479144 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 691/1391] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:48:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:04.551414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b9309-c584-4e9c-a14e-6246695d0d5e None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:04.552263 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b9309-c584-4e9c-a14e-6246695d0d5e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:04.552263 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b9309-c584-4e9c-a14e-6246695d0d5e None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:04.553523 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 701/1392] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:48:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:04.561748 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3900e-b32f-4261-bbe4-d734ff7b7ae8 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:04.562385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3900e-b32f-4261-bbe4-d734ff7b7ae8 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:04.562575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3900e-b32f-4261-bbe4-d734ff7b7ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:04.585013 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-d7a3900e-b32f-4261-bbe4-d734ff7b7ae8 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:04.586260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d7a3900e-b32f-4261-bbe4-d734ff7b7ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:04.600899 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 692/1393] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:48:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:04.648704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b9309-c584-4e9c-a14e-6246695d0d5e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:04.658119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=46 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:48:04.665675 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b9309-c584-4e9c-a14e-6246695d0d5e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:04.671316 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789e247-7ee7-4595-8df7-20da8990998b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:04.671966 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789e247-7ee7-4595-8df7-20da8990998b service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:04.672243 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789e247-7ee7-4595-8df7-20da8990998b service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:04.672522 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789e247-7ee7-4595-8df7-20da8990998b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:04.682440 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789e247-7ee7-4595-8df7-20da8990998b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:04.684162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789e247-7ee7-4595-8df7-20da8990998b service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:04.694549 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 702/1394] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:48:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:06.841606 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96282b9-72a2-429b-9c90-2a3b77ce1759 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:06.842523 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96282b9-72a2-429b-9c90-2a3b77ce1759 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:06.842523 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96282b9-72a2-429b-9c90-2a3b77ce1759 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:06.843592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 693/1395] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:06.849277 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c3e15-e451-4c28-889f-b97dce084da9 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:06.849791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c3e15-e451-4c28-889f-b97dce084da9 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:06.849891 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c3e15-e451-4c28-889f-b97dce084da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:06.869907 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-f57c3e15-e451-4c28-889f-b97dce084da9 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:06.871147 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f57c3e15-e451-4c28-889f-b97dce084da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:06.880664 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 703/1396] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:48:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:48:06.949885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e86c4-4807-4fba-b977-6010c162daf9 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:06.950226 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e86c4-4807-4fba-b977-6010c162daf9 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:06.950467 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e86c4-4807-4fba-b977-6010c162daf9 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:06.951870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 694/1397] 10.222.0.78 () {54 vars in 962 bytes} [Wed May 25 14:48:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:06.958212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd98c40-94ba-42ed-adb0-4c3cca450cb0 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:06.958212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd98c40-94ba-42ed-adb0-4c3cca450cb0 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:06.958492 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd98c40-94ba-42ed-adb0-4c3cca450cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:07.031197 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-fcd98c40-94ba-42ed-adb0-4c3cca450cb0 None None] MFA Rules not processed for user `88cbd76f70d447c4ba92777b2c09821d`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:07.032596 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-fcd98c40-94ba-42ed-adb0-4c3cca450cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:07.119838 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 704/1398] 10.222.0.78 () {58 vars in 1059 bytes} [Wed May 25 14:48:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:07.130376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e86c4-4807-4fba-b977-6010c162daf9 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:07.146271 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e86c4-4807-4fba-b977-6010c162daf9 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:07.150683 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d44aa-5379-4ca0-809c-d0fa368a8d37 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88cbd76f70d447c4ba92777b2c09821d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:07.151332 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d44aa-5379-4ca0-809c-d0fa368a8d37 service glance] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:07.151801 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d44aa-5379-4ca0-809c-d0fa368a8d37 service glance] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:07.152080 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d44aa-5379-4ca0-809c-d0fa368a8d37 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:07.166465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d44aa-5379-4ca0-809c-d0fa368a8d37 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:07.168505 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d44aa-5379-4ca0-809c-d0fa368a8d37 service glance] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:07.180460 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 695/1399] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:48:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:07.200341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed78d6-1dc8-439d-90e3-de3b6624686d None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:07.200751 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed78d6-1dc8-439d-90e3-de3b6624686d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:07.201108 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed78d6-1dc8-439d-90e3-de3b6624686d None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:07.202777 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 705/1400] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:48:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:07.209676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb7945-a264-4aac-8c71-812c87712a5e None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:07.210205 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb7945-a264-4aac-8c71-812c87712a5e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:07.210694 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb7945-a264-4aac-8c71-812c87712a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:07.232935 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:48:07.241522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-b9fb7945-a264-4aac-8c71-812c87712a5e None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:07.243655 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-b9fb7945-a264-4aac-8c71-812c87712a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:07.255821 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 696/1401] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:48:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:07.311917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed78d6-1dc8-439d-90e3-de3b6624686d None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:07.333024 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed78d6-1dc8-439d-90e3-de3b6624686d None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:07.341466 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbaa436-b0b3-429d-9641-93c396cd01c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:07.342747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbaa436-b0b3-429d-9641-93c396cd01c4 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:07.343362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbaa436-b0b3-429d-9641-93c396cd01c4 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:07.343565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbaa436-b0b3-429d-9641-93c396cd01c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:07.356101 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbaa436-b0b3-429d-9641-93c396cd01c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:07.358311 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbaa436-b0b3-429d-9641-93c396cd01c4 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:07.371036 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 706/1402] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:48:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:07.904920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29800076-24d0-4c7c-b75b-229607a6e05c None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:07.906069 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29800076-24d0-4c7c-b75b-229607a6e05c None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:07.906435 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29800076-24d0-4c7c-b75b-229607a6e05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:07.979743 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-29800076-24d0-4c7c-b75b-229607a6e05c None None] MFA Rules not processed for user `b2935579c8554bed8b1e211c47548ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:07.981246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-29800076-24d0-4c7c-b75b-229607a6e05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:08.072695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 697/1403] 10.222.0.78 () {58 vars in 1009 bytes} [Wed May 25 14:48:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:08.127350 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d185a8-997c-4ba6-9df5-dc91c58cad19 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:08.129910 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d185a8-997c-4ba6-9df5-dc91c58cad19 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:08.129910 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d185a8-997c-4ba6-9df5-dc91c58cad19 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:08.130835 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 707/1404] 10.222.0.78 () {54 vars in 942 bytes} [Wed May 25 14:48:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:08.140214 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4576529-f61a-437d-859f-0c9fafeb7b7d None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:08.140508 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4576529-f61a-437d-859f-0c9fafeb7b7d None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:08.140788 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4576529-f61a-437d-859f-0c9fafeb7b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:08.216348 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-e4576529-f61a-437d-859f-0c9fafeb7b7d None None] MFA Rules not processed for user `fbe4a2d106d84fc9ac65c046ba675353`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:08.217749 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-e4576529-f61a-437d-859f-0c9fafeb7b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:08.328463 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 698/1405] 10.222.0.78 () {58 vars in 1039 bytes} [Wed May 25 14:48:08 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:48:08.339994 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d185a8-997c-4ba6-9df5-dc91c58cad19 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:08.357629 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d185a8-997c-4ba6-9df5-dc91c58cad19 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:08.365603 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94dc59-f151-423e-b008-f33b758d10e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe4a2d106d84fc9ac65c046ba675353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:08.366457 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94dc59-f151-423e-b008-f33b758d10e9 service swift] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:08.366740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94dc59-f151-423e-b008-f33b758d10e9 service swift] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:08.367017 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94dc59-f151-423e-b008-f33b758d10e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:08.377733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94dc59-f151-423e-b008-f33b758d10e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:08.380466 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94dc59-f151-423e-b008-f33b758d10e9 service swift] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:08.394167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 708/1406] 10.222.0.78 () {60 vars in 1384 bytes} [Wed May 25 14:48:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 14:48:10.087329 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e70d9-ba7c-41ba-8265-a906a3983e1b None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:10.087329 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e70d9-ba7c-41ba-8265-a906a3983e1b None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:10.087329 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e70d9-ba7c-41ba-8265-a906a3983e1b None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:10.088674 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 699/1407] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:10.094773 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507a1b3-3fb0-4933-a086-bea0eb814af4 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:10.095262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507a1b3-3fb0-4933-a086-bea0eb814af4 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:10.095345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507a1b3-3fb0-4933-a086-bea0eb814af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:10.115668 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-7507a1b3-3fb0-4933-a086-bea0eb814af4 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:10.117061 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7507a1b3-3fb0-4933-a086-bea0eb814af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:10.138073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 709/1408] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:48:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:10.189583 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e70d9-ba7c-41ba-8265-a906a3983e1b None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:10.215984 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e70d9-ba7c-41ba-8265-a906a3983e1b None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:10.227244 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467a9eb-5c80-4ea4-9fcd-cc284faa5c5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:10.228051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467a9eb-5c80-4ea4-9fcd-cc284faa5c5d service nova] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:10.228394 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467a9eb-5c80-4ea4-9fcd-cc284faa5c5d service nova] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:10.228730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467a9eb-5c80-4ea4-9fcd-cc284faa5c5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:10.245663 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9467a9eb-5c80-4ea4-9fcd-cc284faa5c5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:10.247642 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9467a9eb-5c80-4ea4-9fcd-cc284faa5c5d service nova] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:10.268068 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 700/1409] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:48:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:48:20.109006 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=61 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:48:20.255150 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=90 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:48:32.291612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27c896-789d-474e-b6e8-e12b17e7256c None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:32.292264 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27c896-789d-474e-b6e8-e12b17e7256c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:32.292264 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27c896-789d-474e-b6e8-e12b17e7256c None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:32.293569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 710/1410] 10.222.0.78 () {54 vars in 918 bytes} [Wed May 25 14:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:32.298901 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072cd71-8619-40ee-9a1f-8087cab8150a None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:32.299430 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072cd71-8619-40ee-9a1f-8087cab8150a None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:32.300098 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072cd71-8619-40ee-9a1f-8087cab8150a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:32.318674 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-9072cd71-8619-40ee-9a1f-8087cab8150a None None] MFA Rules not processed for user `63789c0a113447ff90933d447cf5bac3`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:32.319872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-9072cd71-8619-40ee-9a1f-8087cab8150a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:32.328334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 701/1411] 10.222.0.78 () {58 vars in 1015 bytes} [Wed May 25 14:48:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:32.343180 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ce0df-4111-48dd-903f-0bc35579e827 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:32.343490 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ce0df-4111-48dd-903f-0bc35579e827 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:32.343751 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ce0df-4111-48dd-903f-0bc35579e827 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:32.345125 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 711/1412] 10.222.0.78 () {54 vars in 918 bytes} [Wed May 25 14:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:32.350219 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa44338-3c32-4e04-808f-ae359093f031 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:32.350504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa44338-3c32-4e04-808f-ae359093f031 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:32.350777 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa44338-3c32-4e04-808f-ae359093f031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:32.368910 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-2aa44338-3c32-4e04-808f-ae359093f031 None None] MFA Rules not processed for user `e34e1d7ffbac49b0bde45829af0c6925`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:32.370156 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-2aa44338-3c32-4e04-808f-ae359093f031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:32.379442 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 702/1413] 10.222.0.78 () {58 vars in 1015 bytes} [Wed May 25 14:48:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:48:35.119577 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb66f8-aca6-416d-ad01-5c74daa56bf8 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:35.120148 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb66f8-aca6-416d-ad01-5c74daa56bf8 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:35.120148 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb66f8-aca6-416d-ad01-5c74daa56bf8 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:35.121515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:35.127739 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eddf5b-db21-4f5b-8a38-5576d6f805a8 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:35.128188 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eddf5b-db21-4f5b-8a38-5576d6f805a8 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:35.128319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eddf5b-db21-4f5b-8a38-5576d6f805a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:35.145978 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-60eddf5b-db21-4f5b-8a38-5576d6f805a8 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:35.147205 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-60eddf5b-db21-4f5b-8a38-5576d6f805a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:35.155106 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:48:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:35.199163 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb66f8-aca6-416d-ad01-5c74daa56bf8 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:35.217937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb66f8-aca6-416d-ad01-5c74daa56bf8 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:35.224931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a7485-dc4c-49a0-a8d7-5445a74d03e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:35.225768 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a7485-dc4c-49a0-a8d7-5445a74d03e7 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:35.225886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a7485-dc4c-49a0-a8d7-5445a74d03e7 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:35.226157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a7485-dc4c-49a0-a8d7-5445a74d03e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:35.235930 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a7485-dc4c-49a0-a8d7-5445a74d03e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:35.238169 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a7485-dc4c-49a0-a8d7-5445a74d03e7 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:35.247897 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 713/1416] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:48:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:36.220130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ee626-086b-48aa-953c-2b4f42f5967e None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:36.220659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ee626-086b-48aa-953c-2b4f42f5967e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:36.220659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ee626-086b-48aa-953c-2b4f42f5967e None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:36.222668 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:48:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:36.227804 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fd87b-09d7-438b-ab3c-4f70e7e7c790 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:36.228234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fd87b-09d7-438b-ab3c-4f70e7e7c790 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:36.228373 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fd87b-09d7-438b-ab3c-4f70e7e7c790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:36.246729 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-466fd87b-09d7-438b-ab3c-4f70e7e7c790 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:36.247959 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-466fd87b-09d7-438b-ab3c-4f70e7e7c790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:36.256569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:48:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:36.448289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ee626-086b-48aa-953c-2b4f42f5967e None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:36.463738 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ee626-086b-48aa-953c-2b4f42f5967e None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:36.469528 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3ed76-b214-46ed-a3be-701f72117e3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:36.470502 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3ed76-b214-46ed-a3be-701f72117e3a service nova] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:36.470502 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3ed76-b214-46ed-a3be-701f72117e3a service nova] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:36.470722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3ed76-b214-46ed-a3be-701f72117e3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:36.480531 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3ed76-b214-46ed-a3be-701f72117e3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:36.482150 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3ed76-b214-46ed-a3be-701f72117e3a service nova] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:36.492620 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 705/1419] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:48:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:37.439095 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e91dfc-9740-4f25-8ddf-df03873b18ca None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:37.439632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e91dfc-9740-4f25-8ddf-df03873b18ca None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:37.439748 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e91dfc-9740-4f25-8ddf-df03873b18ca None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:37.442711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:48:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:37.447939 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c733a-fb9e-4231-8598-3cdf10f8901e None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:37.448463 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c733a-fb9e-4231-8598-3cdf10f8901e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:37.448463 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c733a-fb9e-4231-8598-3cdf10f8901e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:37.466980 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-296c733a-fb9e-4231-8598-3cdf10f8901e None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:37.468198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-296c733a-fb9e-4231-8598-3cdf10f8901e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:37.476732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:48:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:48:37.634854 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e91dfc-9740-4f25-8ddf-df03873b18ca None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:37.649553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e91dfc-9740-4f25-8ddf-df03873b18ca None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:37.655118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca702bc0-c260-425f-91cb-d8bad75d1f22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:37.656017 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca702bc0-c260-425f-91cb-d8bad75d1f22 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:37.656297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca702bc0-c260-425f-91cb-d8bad75d1f22 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:37.656574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca702bc0-c260-425f-91cb-d8bad75d1f22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:37.664884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca702bc0-c260-425f-91cb-d8bad75d1f22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:37.666590 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca702bc0-c260-425f-91cb-d8bad75d1f22 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:37.676457 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 716/1422] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:48:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:38.595714 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7311ace-ef25-40dd-85c4-72523b1d9028 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:38.596341 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7311ace-ef25-40dd-85c4-72523b1d9028 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:38.596341 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7311ace-ef25-40dd-85c4-72523b1d9028 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:38.597689 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:38.603362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0e224-9a66-4653-9d22-ef27c2b51c42 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:38.603896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0e224-9a66-4653-9d22-ef27c2b51c42 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:38.603896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0e224-9a66-4653-9d22-ef27c2b51c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:38.623356 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-46e0e224-9a66-4653-9d22-ef27c2b51c42 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:38.624555 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-46e0e224-9a66-4653-9d22-ef27c2b51c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:38.633096 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:48:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:38.782376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7311ace-ef25-40dd-85c4-72523b1d9028 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:38.797453 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7311ace-ef25-40dd-85c4-72523b1d9028 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:38.803505 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06f581-f1d4-4c74-9e81-720a307766ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:38.804172 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06f581-f1d4-4c74-9e81-720a307766ce service nova] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:38.804452 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06f581-f1d4-4c74-9e81-720a307766ce service nova] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:38.804780 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06f581-f1d4-4c74-9e81-720a307766ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:38.813008 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f06f581-f1d4-4c74-9e81-720a307766ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:38.814637 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f06f581-f1d4-4c74-9e81-720a307766ce service nova] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:38.825239 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 708/1425] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:48:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:39.769261 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3fe94-5358-45c1-8fd4-0c68ddf29abc None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:39.769824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3fe94-5358-45c1-8fd4-0c68ddf29abc None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:39.769919 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3fe94-5358-45c1-8fd4-0c68ddf29abc None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:39.771313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:48:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:39.777918 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e4f18-c306-4186-80f5-7b193a7e10ef None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:39.778523 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e4f18-c306-4186-80f5-7b193a7e10ef None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:39.778523 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e4f18-c306-4186-80f5-7b193a7e10ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:39.797853 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-6e1e4f18-c306-4186-80f5-7b193a7e10ef None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:39.799120 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-6e1e4f18-c306-4186-80f5-7b193a7e10ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:39.807359 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:48:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:39.961038 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3fe94-5358-45c1-8fd4-0c68ddf29abc None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:39.976400 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3fe94-5358-45c1-8fd4-0c68ddf29abc None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:39.981866 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b96d6-dcea-4363-93fb-7c8b8ee517a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:39.982538 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b96d6-dcea-4363-93fb-7c8b8ee517a6 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:39.982824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b96d6-dcea-4363-93fb-7c8b8ee517a6 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:39.983104 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b96d6-dcea-4363-93fb-7c8b8ee517a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:39.992614 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b96d6-dcea-4363-93fb-7c8b8ee517a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:39.994260 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b96d6-dcea-4363-93fb-7c8b8ee517a6 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:40.004402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 719/1428] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:48:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:40.892547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a201b7-d8c3-4c49-84ea-7420f37e9f61 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:40.893758 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a201b7-d8c3-4c49-84ea-7420f37e9f61 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:40.893758 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a201b7-d8c3-4c49-84ea-7420f37e9f61 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:40.895067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:48:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:40.900468 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c54d1-ad9e-47ef-aeac-e078a5eb312b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:40.900977 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c54d1-ad9e-47ef-aeac-e078a5eb312b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:40.901079 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c54d1-ad9e-47ef-aeac-e078a5eb312b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:40.920372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-704c54d1-ad9e-47ef-aeac-e078a5eb312b None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:40.921592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-704c54d1-ad9e-47ef-aeac-e078a5eb312b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:40.930342 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:48:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:41.082161 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a201b7-d8c3-4c49-84ea-7420f37e9f61 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:41.096492 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a201b7-d8c3-4c49-84ea-7420f37e9f61 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:41.102131 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12a53d-6134-4dc1-b426-f3e42747b014 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:41.102777 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12a53d-6134-4dc1-b426-f3e42747b014 service nova] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:41.103059 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12a53d-6134-4dc1-b426-f3e42747b014 service nova] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:41.103337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12a53d-6134-4dc1-b426-f3e42747b014 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:41.112380 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12a53d-6134-4dc1-b426-f3e42747b014 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:41.113974 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12a53d-6134-4dc1-b426-f3e42747b014 service nova] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:41.123742 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 711/1431] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:48:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:41.992141 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddefb6-7333-4b8b-9c1a-a66ba3c8943c None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:41.992660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddefb6-7333-4b8b-9c1a-a66ba3c8943c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:41.992775 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddefb6-7333-4b8b-9c1a-a66ba3c8943c None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:41.994082 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:41.999022 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d645151-1188-4bc5-b765-b7583d29f062 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:41.999604 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d645151-1188-4bc5-b765-b7583d29f062 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:41.999604 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d645151-1188-4bc5-b765-b7583d29f062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:42.018809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-4d645151-1188-4bc5-b765-b7583d29f062 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:42.021022 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-4d645151-1188-4bc5-b765-b7583d29f062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:42.027918 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:42.180002 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceddefb6-7333-4b8b-9c1a-a66ba3c8943c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:42.195049 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceddefb6-7333-4b8b-9c1a-a66ba3c8943c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:42.200418 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba7d01-a371-459d-b66f-e8b9ab56e3dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:42.201060 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba7d01-a371-459d-b66f-e8b9ab56e3dd service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:42.201345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba7d01-a371-459d-b66f-e8b9ab56e3dd service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:42.201626 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba7d01-a371-459d-b66f-e8b9ab56e3dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:42.211098 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba7d01-a371-459d-b66f-e8b9ab56e3dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:42.212704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba7d01-a371-459d-b66f-e8b9ab56e3dd service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:42.223379 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 722/1434] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:48:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:43.079042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85779ffc-723a-411a-b9e5-ae76bc713494 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:43.079606 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85779ffc-723a-411a-b9e5-ae76bc713494 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:43.079606 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85779ffc-723a-411a-b9e5-ae76bc713494 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:43.081619 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:48:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:43.087017 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727be403-0a4f-4859-8fe3-36a36efa8045 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:43.087763 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727be403-0a4f-4859-8fe3-36a36efa8045 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:43.087978 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727be403-0a4f-4859-8fe3-36a36efa8045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:43.107570 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-727be403-0a4f-4859-8fe3-36a36efa8045 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:43.108792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-727be403-0a4f-4859-8fe3-36a36efa8045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:43.116814 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:43.269792 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85779ffc-723a-411a-b9e5-ae76bc713494 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:43.285306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85779ffc-723a-411a-b9e5-ae76bc713494 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:43.290964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d1aa9-4f0e-4694-a2d7-5d729d10a119 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:43.291636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d1aa9-4f0e-4694-a2d7-5d729d10a119 service nova] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:43.291919 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d1aa9-4f0e-4694-a2d7-5d729d10a119 service nova] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:43.292196 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d1aa9-4f0e-4694-a2d7-5d729d10a119 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:43.301369 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898d1aa9-4f0e-4694-a2d7-5d729d10a119 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:43.303053 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898d1aa9-4f0e-4694-a2d7-5d729d10a119 service nova] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:43.313625 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 714/1437] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:48:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:44.196902 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbced93-886c-42f9-9d14-308d5ec8d14b None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:44.197439 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbced93-886c-42f9-9d14-308d5ec8d14b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:44.197439 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbced93-886c-42f9-9d14-308d5ec8d14b None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:44.198818 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:44.204508 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c893b-2b9f-4cb7-b634-c822aca39e81 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:44.205013 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c893b-2b9f-4cb7-b634-c822aca39e81 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:44.205124 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c893b-2b9f-4cb7-b634-c822aca39e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:44.223382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-c58c893b-2b9f-4cb7-b634-c822aca39e81 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:44.224566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-c58c893b-2b9f-4cb7-b634-c822aca39e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:44.232450 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:44.383018 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbced93-886c-42f9-9d14-308d5ec8d14b None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:44.398418 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbced93-886c-42f9-9d14-308d5ec8d14b None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:44.403845 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a99484-10c9-4588-955c-11e5def4683d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:44.404518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a99484-10c9-4588-955c-11e5def4683d service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:44.404805 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a99484-10c9-4588-955c-11e5def4683d service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:44.405081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a99484-10c9-4588-955c-11e5def4683d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:44.414515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a99484-10c9-4588-955c-11e5def4683d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:44.416116 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a99484-10c9-4588-955c-11e5def4683d service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:44.425874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 725/1440] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:48:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:46.581850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e20b7-d89e-48ea-9592-86a5e80712cd None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:46.582433 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e20b7-d89e-48ea-9592-86a5e80712cd None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:46.582433 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e20b7-d89e-48ea-9592-86a5e80712cd None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:46.583848 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:48:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:46.589866 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19b6df-97ab-4797-8001-1fd6ba77a2aa None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:46.590386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19b6df-97ab-4797-8001-1fd6ba77a2aa None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:46.590497 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19b6df-97ab-4797-8001-1fd6ba77a2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:46.608464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-4b19b6df-97ab-4797-8001-1fd6ba77a2aa None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:46.609684 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4b19b6df-97ab-4797-8001-1fd6ba77a2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:46.618284 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:46.657015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e20b7-d89e-48ea-9592-86a5e80712cd None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:46.658819 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-c12e20b7-d89e-48ea-9592-86a5e80712cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:46.671841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e20b7-d89e-48ea-9592-86a5e80712cd None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:46.675844 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42961b-5ecc-4cb2-9fe8-82cc10d4e80b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:46.676489 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42961b-5ecc-4cb2-9fe8-82cc10d4e80b service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:46.676766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42961b-5ecc-4cb2-9fe8-82cc10d4e80b service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:46.677037 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42961b-5ecc-4cb2-9fe8-82cc10d4e80b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:46.686484 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42961b-5ecc-4cb2-9fe8-82cc10d4e80b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:46.687921 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42961b-5ecc-4cb2-9fe8-82cc10d4e80b service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:46.698871 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 717/1443] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:48:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:49.150752 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74542d3c-df3b-48c3-b0a9-d02cb9bed09b None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:49.151316 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74542d3c-df3b-48c3-b0a9-d02cb9bed09b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:49.151316 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74542d3c-df3b-48c3-b0a9-d02cb9bed09b None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:49.152684 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:49.158391 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba364cc2-6843-4ce9-bc14-7fb663956d41 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:49.159456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba364cc2-6843-4ce9-bc14-7fb663956d41 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:49.159567 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba364cc2-6843-4ce9-bc14-7fb663956d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:49.177430 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-ba364cc2-6843-4ce9-bc14-7fb663956d41 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:49.178640 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-ba364cc2-6843-4ce9-bc14-7fb663956d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:49.186629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:48:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:49.224531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74542d3c-df3b-48c3-b0a9-d02cb9bed09b None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:49.236331 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74542d3c-df3b-48c3-b0a9-d02cb9bed09b None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:49.240858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204563e-ca42-4249-b908-45e9507365f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:49.241516 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204563e-ca42-4249-b908-45e9507365f8 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:49.241800 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204563e-ca42-4249-b908-45e9507365f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:49.242117 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204563e-ca42-4249-b908-45e9507365f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:49.254685 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4204563e-ca42-4249-b908-45e9507365f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:49.257656 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4204563e-ca42-4249-b908-45e9507365f8 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:49.275235 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 728/1446] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:48:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:51.856939 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9ac18-c79d-4c05-8d80-60f08db9ba67 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:51.857928 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9ac18-c79d-4c05-8d80-60f08db9ba67 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:51.857928 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9ac18-c79d-4c05-8d80-60f08db9ba67 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:51.859739 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:48:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:51.865902 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96af3a3-d6c0-416b-94e6-6780114d5401 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:51.866449 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96af3a3-d6c0-416b-94e6-6780114d5401 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:51.866555 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96af3a3-d6c0-416b-94e6-6780114d5401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:51.889979 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-e96af3a3-d6c0-416b-94e6-6780114d5401 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:51.891227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e96af3a3-d6c0-416b-94e6-6780114d5401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:51.899902 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:48:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:51.938794 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9ac18-c79d-4c05-8d80-60f08db9ba67 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:51.951117 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9ac18-c79d-4c05-8d80-60f08db9ba67 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:51.955962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea73d0-8af3-4258-9352-b119533bd485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:51.956887 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea73d0-8af3-4258-9352-b119533bd485 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:51.957287 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea73d0-8af3-4258-9352-b119533bd485 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:51.957672 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea73d0-8af3-4258-9352-b119533bd485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:51.968437 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea73d0-8af3-4258-9352-b119533bd485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:51.969919 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea73d0-8af3-4258-9352-b119533bd485 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:51.979713 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 720/1449] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:48:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:54.378641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5643a-409f-4ded-aa12-db8074002906 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:54.379235 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5643a-409f-4ded-aa12-db8074002906 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:54.379235 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5643a-409f-4ded-aa12-db8074002906 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:54.380582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:48:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:54.386096 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3dfeeb-7693-4328-950e-4ca5a37205f3 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:54.386581 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3dfeeb-7693-4328-950e-4ca5a37205f3 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:54.386687 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3dfeeb-7693-4328-950e-4ca5a37205f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:54.404180 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-1d3dfeeb-7693-4328-950e-4ca5a37205f3 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:54.405658 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-1d3dfeeb-7693-4328-950e-4ca5a37205f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:54.413372 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:48:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:54.452310 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5643a-409f-4ded-aa12-db8074002906 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:54.462958 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5643a-409f-4ded-aa12-db8074002906 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:54.466530 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed5802-ab0d-42ef-b802-01f3d3d98eff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:54.467158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed5802-ab0d-42ef-b802-01f3d3d98eff service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:54.467438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed5802-ab0d-42ef-b802-01f3d3d98eff service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:54.467710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed5802-ab0d-42ef-b802-01f3d3d98eff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:54.477148 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed5802-ab0d-42ef-b802-01f3d3d98eff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:54.478611 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed5802-ab0d-42ef-b802-01f3d3d98eff service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:54.488535 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 731/1452] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:48:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:56.895781 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde72ad-f9ab-4e1c-aba2-e2efe9bc46d4 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:56.896321 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde72ad-f9ab-4e1c-aba2-e2efe9bc46d4 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:56.896321 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde72ad-f9ab-4e1c-aba2-e2efe9bc46d4 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:56.897786 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:48:56.903772 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ac213-3551-49ef-9251-c0d2bd6dcee2 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:56.904249 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ac213-3551-49ef-9251-c0d2bd6dcee2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:56.904355 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ac213-3551-49ef-9251-c0d2bd6dcee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:56.922367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b60ac213-3551-49ef-9251-c0d2bd6dcee2 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:56.923575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b60ac213-3551-49ef-9251-c0d2bd6dcee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:56.931535 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:48:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:56.970262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde72ad-f9ab-4e1c-aba2-e2efe9bc46d4 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:56.981523 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde72ad-f9ab-4e1c-aba2-e2efe9bc46d4 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:56.985067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a77fc-b396-45c8-ba11-dc9e7ec068aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:56.985723 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a77fc-b396-45c8-ba11-dc9e7ec068aa service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:56.986038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a77fc-b396-45c8-ba11-dc9e7ec068aa service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:56.986313 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a77fc-b396-45c8-ba11-dc9e7ec068aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:56.997310 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a77fc-b396-45c8-ba11-dc9e7ec068aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:56.998826 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a77fc-b396-45c8-ba11-dc9e7ec068aa service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:57.009853 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 723/1455] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:48:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:59.345473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15937e-d3a3-4c22-9a99-005f75949b96 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:59.346681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15937e-d3a3-4c22-9a99-005f75949b96 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:59.346681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15937e-d3a3-4c22-9a99-005f75949b96 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:59.348078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:48:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:48:59.354634 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2e7da-ea7e-4e24-8403-987a4d769824 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:59.355193 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2e7da-ea7e-4e24-8403-987a4d769824 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:59.355193 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2e7da-ea7e-4e24-8403-987a4d769824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:59.374707 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-cbb2e7da-ea7e-4e24-8403-987a4d769824 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:48:59.375911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-cbb2e7da-ea7e-4e24-8403-987a4d769824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:59.384718 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:48:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:48:59.423113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15937e-d3a3-4c22-9a99-005f75949b96 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:59.433828 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15937e-d3a3-4c22-9a99-005f75949b96 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:59.437398 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b688b-544b-4e89-b145-327eefa4041a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:59.438058 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b688b-544b-4e89-b145-327eefa4041a service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:59.438347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b688b-544b-4e89-b145-327eefa4041a service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:59.438626 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b688b-544b-4e89-b145-327eefa4041a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:59.447952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93b688b-544b-4e89-b145-327eefa4041a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:59.449412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93b688b-544b-4e89-b145-327eefa4041a service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:59.460052 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 734/1458] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:48:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:49:01.770656 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a7d61-ddc8-4ae6-a10c-ee10a12ef18c None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:01.771226 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a7d61-ddc8-4ae6-a10c-ee10a12ef18c None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:01.771226 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a7d61-ddc8-4ae6-a10c-ee10a12ef18c None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:01.772564 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:49:01.777931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18762773-bca4-455c-b2d2-7b5a4c782439 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:01.778438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18762773-bca4-455c-b2d2-7b5a4c782439 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:01.778559 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18762773-bca4-455c-b2d2-7b5a4c782439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:01.784805 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=70 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:49:01.798256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-18762773-bca4-455c-b2d2-7b5a4c782439 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:49:01.800008 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-18762773-bca4-455c-b2d2-7b5a4c782439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:49:01.807958 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:49:01.845489 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a7d61-ddc8-4ae6-a10c-ee10a12ef18c None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:49:01.851067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=73 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:49:01.857642 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a7d61-ddc8-4ae6-a10c-ee10a12ef18c None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:49:01.861261 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fbd6a4-5c00-4ebd-90b1-3a817f1886ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:49:01.862442 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fbd6a4-5c00-4ebd-90b1-3a817f1886ba service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:01.862726 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fbd6a4-5c00-4ebd-90b1-3a817f1886ba service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:01.863001 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fbd6a4-5c00-4ebd-90b1-3a817f1886ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:01.873173 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fbd6a4-5c00-4ebd-90b1-3a817f1886ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:49:01.874652 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fbd6a4-5c00-4ebd-90b1-3a817f1886ba service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:49:01.885998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 726/1461] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:49:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:49:04.236200 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593a281-cc60-4ea7-87f1-d1e3a027a639 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:04.237374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593a281-cc60-4ea7-87f1-d1e3a027a639 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:04.237374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593a281-cc60-4ea7-87f1-d1e3a027a639 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:04.238872 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:49:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:49:04.244848 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd82400-a48a-4f0e-a69a-66d119f94c3c None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:04.245522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd82400-a48a-4f0e-a69a-66d119f94c3c None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:04.245522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd82400-a48a-4f0e-a69a-66d119f94c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:04.263757 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-ffd82400-a48a-4f0e-a69a-66d119f94c3c None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:49:04.264984 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-ffd82400-a48a-4f0e-a69a-66d119f94c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:49:04.273311 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:49:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:49:04.313947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593a281-cc60-4ea7-87f1-d1e3a027a639 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:49:04.327596 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593a281-cc60-4ea7-87f1-d1e3a027a639 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:49:04.332259 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a246ed5-acbd-4c56-a3e8-51f8bbd4963b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:49:04.332916 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a246ed5-acbd-4c56-a3e8-51f8bbd4963b service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:04.333201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a246ed5-acbd-4c56-a3e8-51f8bbd4963b service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:04.333509 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a246ed5-acbd-4c56-a3e8-51f8bbd4963b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:04.343345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a246ed5-acbd-4c56-a3e8-51f8bbd4963b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:49:04.344828 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a246ed5-acbd-4c56-a3e8-51f8bbd4963b service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:49:04.356035 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 737/1464] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:49:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:49:06.718163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f32c8-eaa7-4564-acff-7d6699a71769 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:06.718707 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f32c8-eaa7-4564-acff-7d6699a71769 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:06.718707 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f32c8-eaa7-4564-acff-7d6699a71769 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:06.720111 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:49:06.726226 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5f605-5c45-43b0-a296-e177ad879ea7 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:06.726706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5f605-5c45-43b0-a296-e177ad879ea7 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:06.726803 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5f605-5c45-43b0-a296-e177ad879ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:06.747598 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-d2f5f605-5c45-43b0-a296-e177ad879ea7 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:49:06.748822 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d2f5f605-5c45-43b0-a296-e177ad879ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:49:06.757231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:49:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:49:06.794265 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f32c8-eaa7-4564-acff-7d6699a71769 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:49:06.805727 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f32c8-eaa7-4564-acff-7d6699a71769 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:49:06.809245 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd24b6-d076-4825-9a74-8af7e6c44d17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:49:06.809888 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd24b6-d076-4825-9a74-8af7e6c44d17 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:06.810195 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd24b6-d076-4825-9a74-8af7e6c44d17 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:06.810466 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd24b6-d076-4825-9a74-8af7e6c44d17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:06.823090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd24b6-d076-4825-9a74-8af7e6c44d17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:49:06.824533 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd24b6-d076-4825-9a74-8af7e6c44d17 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:49:06.835172 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 729/1467] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:49:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:49:09.121248 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea20721-d368-497c-91c3-7bdd5fa980a4 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:09.121835 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea20721-d368-497c-91c3-7bdd5fa980a4 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:09.121835 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea20721-d368-497c-91c3-7bdd5fa980a4 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:09.123201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:49:09.128749 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d58f7b-ec4a-463c-85ea-11adb28e915e None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:09.129224 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d58f7b-ec4a-463c-85ea-11adb28e915e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:09.129332 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d58f7b-ec4a-463c-85ea-11adb28e915e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:09.147761 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-f0d58f7b-ec4a-463c-85ea-11adb28e915e None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:49:09.148968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f0d58f7b-ec4a-463c-85ea-11adb28e915e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:49:09.157247 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:49:09.196220 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea20721-d368-497c-91c3-7bdd5fa980a4 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:49:09.207538 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea20721-d368-497c-91c3-7bdd5fa980a4 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:49:09.211114 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c03f5-efa2-47a7-9d01-2ac8ec79c99f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:49:09.211765 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c03f5-efa2-47a7-9d01-2ac8ec79c99f service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:09.212046 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c03f5-efa2-47a7-9d01-2ac8ec79c99f service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:09.212320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c03f5-efa2-47a7-9d01-2ac8ec79c99f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:09.225575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c03f5-efa2-47a7-9d01-2ac8ec79c99f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:49:09.227167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c03f5-efa2-47a7-9d01-2ac8ec79c99f service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:49:09.238715 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 740/1470] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:49:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:49:19.141752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=17 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:49:19.233328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=17 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:49:43.659228 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd72a5-f38f-4129-a699-edb455d3c111 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:43.659868 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd72a5-f38f-4129-a699-edb455d3c111 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:43.659868 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd72a5-f38f-4129-a699-edb455d3c111 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:43.661146 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 731/1471] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:49:43.666454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140fe95-ba04-4bb3-b3f6-e30902e99e96 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:43.666904 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140fe95-ba04-4bb3-b3f6-e30902e99e96 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:43.667015 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140fe95-ba04-4bb3-b3f6-e30902e99e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:43.685054 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-1140fe95-ba04-4bb3-b3f6-e30902e99e96 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:49:43.686299 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1140fe95-ba04-4bb3-b3f6-e30902e99e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:49:43.694941 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 741/1472] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:49:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:49:43.741535 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a9b23-1abd-452c-9a40-6f5cb1a38663 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:43.741824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a9b23-1abd-452c-9a40-6f5cb1a38663 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:43.742141 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a9b23-1abd-452c-9a40-6f5cb1a38663 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:43.743512 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 732/1473] 10.222.0.78 () {54 vars in 962 bytes} [Wed May 25 14:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:49:43.748383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051219aa-f2f9-40b9-b22f-30ac57d8e597 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:43.748673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051219aa-f2f9-40b9-b22f-30ac57d8e597 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:43.748955 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051219aa-f2f9-40b9-b22f-30ac57d8e597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:43.767740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-051219aa-f2f9-40b9-b22f-30ac57d8e597 None None] MFA Rules not processed for user `88cbd76f70d447c4ba92777b2c09821d`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:49:43.768949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-051219aa-f2f9-40b9-b22f-30ac57d8e597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:49:43.775944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 742/1474] 10.222.0.78 () {58 vars in 1059 bytes} [Wed May 25 14:49:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:49:43.783792 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a9b23-1abd-452c-9a40-6f5cb1a38663 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:49:43.795625 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a9b23-1abd-452c-9a40-6f5cb1a38663 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:49:43.799742 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdfc15e-d42f-44ee-a803-4e657d0aae6b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88cbd76f70d447c4ba92777b2c09821d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:49:43.800381 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfc15e-d42f-44ee-a803-4e657d0aae6b service glance] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:43.800665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfc15e-d42f-44ee-a803-4e657d0aae6b service glance] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:43.800934 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfc15e-d42f-44ee-a803-4e657d0aae6b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:43.810826 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdfc15e-d42f-44ee-a803-4e657d0aae6b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:49:43.812350 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdfc15e-d42f-44ee-a803-4e657d0aae6b service glance] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:49:43.822994 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 733/1475] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:49:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:49:46.151966 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d7282-4e39-4246-a34c-f9c88a0fb6a6 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:46.152533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d7282-4e39-4246-a34c-f9c88a0fb6a6 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:46.152533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d7282-4e39-4246-a34c-f9c88a0fb6a6 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:46.153917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 743/1476] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:49:46.161156 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb237c-9d9d-4a94-9bb4-33af05fd6625 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:46.161655 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb237c-9d9d-4a94-9bb4-33af05fd6625 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:46.161655 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb237c-9d9d-4a94-9bb4-33af05fd6625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:46.179490 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-82fb237c-9d9d-4a94-9bb4-33af05fd6625 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:49:46.180700 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-82fb237c-9d9d-4a94-9bb4-33af05fd6625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:49:46.189329 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 734/1477] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:49:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:49:46.213337 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d7282-4e39-4246-a34c-f9c88a0fb6a6 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:49:46.226773 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d7282-4e39-4246-a34c-f9c88a0fb6a6 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:49:46.231914 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb70d27-cfe5-466a-a635-dcda76d75bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:49:46.232577 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb70d27-cfe5-466a-a635-dcda76d75bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:46.232863 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb70d27-cfe5-466a-a635-dcda76d75bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:46.233178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb70d27-cfe5-466a-a635-dcda76d75bd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:46.233942 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb70d27-cfe5-466a-a635-dcda76d75bd4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:49:46.236461 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: WARNING py.warnings [None req-9bb70d27-cfe5-466a-a635-dcda76d75bd4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:49:46.236461 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: warnings.warn(msg) May 25 14:49:46.236461 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: May 25 14:49:46.237625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb70d27-cfe5-466a-a635-dcda76d75bd4 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:49:46.257722 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 744/1478] 10.222.0.78 () {56 vars in 1108 bytes} [Wed May 25 14:49:46 2022] GET /identity/v3/projects => generated 2553 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:49:48.522463 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d3a66-3b0f-4eb5-8774-6ab7f3c0f1b2 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:48.523309 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d3a66-3b0f-4eb5-8774-6ab7f3c0f1b2 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:48.523309 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d3a66-3b0f-4eb5-8774-6ab7f3c0f1b2 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:48.524395 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 735/1479] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:49:48.530392 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ca0dc-0e1a-407b-b624-938f494f1780 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:48.530878 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ca0dc-0e1a-407b-b624-938f494f1780 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:48.531007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ca0dc-0e1a-407b-b624-938f494f1780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:48.551086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b25ca0dc-0e1a-407b-b624-938f494f1780 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:49:48.552333 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b25ca0dc-0e1a-407b-b624-938f494f1780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:49:48.560469 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 745/1480] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:49:48.605884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d3a66-3b0f-4eb5-8774-6ab7f3c0f1b2 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:49:48.620336 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d3a66-3b0f-4eb5-8774-6ab7f3c0f1b2 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:49:48.626431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4230290-128c-47ad-bf3d-b769b08ed0de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:49:48.626984 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4230290-128c-47ad-bf3d-b769b08ed0de service nova] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:48.627259 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4230290-128c-47ad-bf3d-b769b08ed0de service nova] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:48.627529 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4230290-128c-47ad-bf3d-b769b08ed0de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:48.636470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4230290-128c-47ad-bf3d-b769b08ed0de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:49:48.638091 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4230290-128c-47ad-bf3d-b769b08ed0de service nova] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:49:48.648129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 736/1481] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:49:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:49:51.110756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b307a-6e2e-4ce4-891b-9cf5014962f2 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:51.111606 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b307a-6e2e-4ce4-891b-9cf5014962f2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:51.111606 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b307a-6e2e-4ce4-891b-9cf5014962f2 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:51.112788 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 746/1482] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:49:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:49:51.118688 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881ee3b-0650-4869-9ce1-c4c2e573ed07 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:51.119189 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881ee3b-0650-4869-9ce1-c4c2e573ed07 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:51.119297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881ee3b-0650-4869-9ce1-c4c2e573ed07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:51.138920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-c881ee3b-0650-4869-9ce1-c4c2e573ed07 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:49:51.141087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-c881ee3b-0650-4869-9ce1-c4c2e573ed07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:49:51.150531 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 737/1483] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:49:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:49:51.187705 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b307a-6e2e-4ce4-891b-9cf5014962f2 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:49:51.200025 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b307a-6e2e-4ce4-891b-9cf5014962f2 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:49:51.203599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2ff76-bc86-4c80-8845-45de88585565 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:49:51.204773 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2ff76-bc86-4c80-8845-45de88585565 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:51.205054 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2ff76-bc86-4c80-8845-45de88585565 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:51.205331 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2ff76-bc86-4c80-8845-45de88585565 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:51.215419 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2ff76-bc86-4c80-8845-45de88585565 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:49:51.216914 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2ff76-bc86-4c80-8845-45de88585565 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:49:51.228632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 747/1484] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:49:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:49:53.539942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e95672-c53c-49c9-8693-6ae5c3886c19 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:53.540592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e95672-c53c-49c9-8693-6ae5c3886c19 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:53.540592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e95672-c53c-49c9-8693-6ae5c3886c19 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:53.541933 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 738/1485] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:49:53.548230 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7893ca-9f0a-4386-be7b-8cfbc68594bd None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:53.548700 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7893ca-9f0a-4386-be7b-8cfbc68594bd None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:53.548700 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7893ca-9f0a-4386-be7b-8cfbc68594bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:53.567292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-1c7893ca-9f0a-4386-be7b-8cfbc68594bd None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:49:53.568514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1c7893ca-9f0a-4386-be7b-8cfbc68594bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:49:53.576432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 748/1486] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:49:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:49:53.615683 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e95672-c53c-49c9-8693-6ae5c3886c19 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:49:53.627349 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e95672-c53c-49c9-8693-6ae5c3886c19 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:49:53.630892 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee11a79-3527-4f67-9cb2-c421fce97b95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:49:53.631533 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee11a79-3527-4f67-9cb2-c421fce97b95 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:53.631812 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee11a79-3527-4f67-9cb2-c421fce97b95 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:53.632087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee11a79-3527-4f67-9cb2-c421fce97b95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:53.642447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee11a79-3527-4f67-9cb2-c421fce97b95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:49:53.643874 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee11a79-3527-4f67-9cb2-c421fce97b95 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:49:53.654120 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 739/1487] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:49:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:49:56.084336 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f3b8f-3d13-421d-8747-1cff31a2ba29 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:56.084895 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f3b8f-3d13-421d-8747-1cff31a2ba29 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:56.084895 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f3b8f-3d13-421d-8747-1cff31a2ba29 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:56.086338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 749/1488] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:49:56.091875 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030947e7-48f7-4abf-ab26-c4888132cb5e None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:56.092344 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030947e7-48f7-4abf-ab26-c4888132cb5e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:56.092445 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030947e7-48f7-4abf-ab26-c4888132cb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:56.111852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-030947e7-48f7-4abf-ab26-c4888132cb5e None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:49:56.113064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-030947e7-48f7-4abf-ab26-c4888132cb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:49:56.121113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 740/1489] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:49:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:49:56.162986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f3b8f-3d13-421d-8747-1cff31a2ba29 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:49:56.175529 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f3b8f-3d13-421d-8747-1cff31a2ba29 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:49:56.179163 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0c932-3785-41af-9e89-b446948b3ab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:49:56.179806 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0c932-3785-41af-9e89-b446948b3ab2 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:56.180086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0c932-3785-41af-9e89-b446948b3ab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:56.180365 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0c932-3785-41af-9e89-b446948b3ab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:56.190167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0c932-3785-41af-9e89-b446948b3ab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:49:56.192376 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0c932-3785-41af-9e89-b446948b3ab2 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:49:56.202642 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 750/1490] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:49:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:49:58.790140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff56f1-bba3-4f70-b607-12bcfd734c30 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:58.790757 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff56f1-bba3-4f70-b607-12bcfd734c30 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:58.790757 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff56f1-bba3-4f70-b607-12bcfd734c30 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:58.792090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 741/1491] 10.222.0.78 () {54 vars in 926 bytes} [Wed May 25 14:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:49:58.798576 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6955ff9-9260-4291-8891-3f3a6b25dc40 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:58.799078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6955ff9-9260-4291-8891-3f3a6b25dc40 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:58.799184 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6955ff9-9260-4291-8891-3f3a6b25dc40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:58.819895 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-e6955ff9-9260-4291-8891-3f3a6b25dc40 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:49:58.821432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e6955ff9-9260-4291-8891-3f3a6b25dc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:49:58.831326 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 751/1492] 10.222.0.78 () {58 vars in 1023 bytes} [Wed May 25 14:49:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:49:58.870072 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff56f1-bba3-4f70-b607-12bcfd734c30 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:49:58.881862 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff56f1-bba3-4f70-b607-12bcfd734c30 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:49:58.885591 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e23e51-af9c-4fac-acbb-db06f74a6258 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:49:58.886310 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e23e51-af9c-4fac-acbb-db06f74a6258 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:58.886610 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e23e51-af9c-4fac-acbb-db06f74a6258 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:58.886913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e23e51-af9c-4fac-acbb-db06f74a6258 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:58.897982 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e23e51-af9c-4fac-acbb-db06f74a6258 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:49:58.899428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e23e51-af9c-4fac-acbb-db06f74a6258 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:49:58.911393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 742/1493] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:49:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:50:08.813632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=33 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:50:08.905626 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=33 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:51:49.733417 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95851ca-8316-483f-aff5-7197c39690bc None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:51:49.734052 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95851ca-8316-483f-aff5-7197c39690bc None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:51:49.734052 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95851ca-8316-483f-aff5-7197c39690bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:51:49.751901 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-e95851ca-8316-483f-aff5-7197c39690bc None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:51:49.753081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e95851ca-8316-483f-aff5-7197c39690bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:51:49.760826 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 752/1494] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:51:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:51:49.770763 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21ca48-552f-4c65-a597-d3372b2646ec None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:51:49.771281 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21ca48-552f-4c65-a597-d3372b2646ec None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:51:49.771281 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21ca48-552f-4c65-a597-d3372b2646ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:51:49.789705 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-4e21ca48-552f-4c65-a597-d3372b2646ec None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:51:49.790931 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-4e21ca48-552f-4c65-a597-d3372b2646ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:51:49.798796 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 743/1495] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:51:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:51:49.803234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95851ca-8316-483f-aff5-7197c39690bc None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:51:49.812837 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95851ca-8316-483f-aff5-7197c39690bc None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:51:49.817309 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b5aa8-51fc-481d-84cc-ab6c7c20f5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:51:49.817953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b5aa8-51fc-481d-84cc-ab6c7c20f5f8 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:51:49.818265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b5aa8-51fc-481d-84cc-ab6c7c20f5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:51:49.818542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b5aa8-51fc-481d-84cc-ab6c7c20f5f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:51:49.819252 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b5aa8-51fc-481d-84cc-ab6c7c20f5f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:51:49.819554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b5aa8-51fc-481d-84cc-ab6c7c20f5f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:51:49.820570 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: WARNING py.warnings [None req-972b5aa8-51fc-481d-84cc-ab6c7c20f5f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 14:51:49.820570 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: warnings.warn(msg) May 25 14:51:49.820570 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: May 25 14:51:49.821454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b5aa8-51fc-481d-84cc-ab6c7c20f5f8 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:51:49.834183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 753/1496] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:51:49.842996 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21ca48-552f-4c65-a597-d3372b2646ec None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:51:49.852325 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21ca48-552f-4c65-a597-d3372b2646ec None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:51:49.857547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4b2cc-d2d7-4269-aaf1-9fe83d17117f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:51:49.858312 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4b2cc-d2d7-4269-aaf1-9fe83d17117f admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:51:49.858592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4b2cc-d2d7-4269-aaf1-9fe83d17117f admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:51:49.858861 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4b2cc-d2d7-4269-aaf1-9fe83d17117f admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:51:49.859590 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4b2cc-d2d7-4269-aaf1-9fe83d17117f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:51:49.861416 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: WARNING py.warnings [None req-eed4b2cc-d2d7-4269-aaf1-9fe83d17117f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:51:49.861416 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: warnings.warn(msg) May 25 14:51:49.861416 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: May 25 14:51:49.862681 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4b2cc-d2d7-4269-aaf1-9fe83d17117f admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:51:49.902523 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 744/1497] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:51:49 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:51:49.907058 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b5aa8-51fc-481d-84cc-ab6c7c20f5f8 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:51:49.919182 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b5aa8-51fc-481d-84cc-ab6c7c20f5f8 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:51:49.982039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a02c8-b2f6-418d-b1b7-7259ec70a40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:51:49.982723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a02c8-b2f6-418d-b1b7-7259ec70a40c admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:51:49.983593 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a02c8-b2f6-418d-b1b7-7259ec70a40c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:51:49.983814 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a02c8-b2f6-418d-b1b7-7259ec70a40c admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:51:49.984525 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a02c8-b2f6-418d-b1b7-7259ec70a40c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:51:49.985595 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: WARNING py.warnings [None req-bc1a02c8-b2f6-418d-b1b7-7259ec70a40c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:51:49.985595 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: warnings.warn(msg) May 25 14:51:49.985595 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: May 25 14:51:49.986861 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a02c8-b2f6-418d-b1b7-7259ec70a40c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:51:50.008644 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 754/1498] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:51:49 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:51:50.012767 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4b2cc-d2d7-4269-aaf1-9fe83d17117f admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:51:50.023033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4b2cc-d2d7-4269-aaf1-9fe83d17117f admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:51:50.028006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c09a59-ec69-41a9-a904-fd95edc4d79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:51:50.029161 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c09a59-ec69-41a9-a904-fd95edc4d79b admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:51:50.029434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c09a59-ec69-41a9-a904-fd95edc4d79b admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:51:50.029734 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c09a59-ec69-41a9-a904-fd95edc4d79b admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:51:50.030495 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c09a59-ec69-41a9-a904-fd95edc4d79b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:51:50.031506 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: WARNING py.warnings [None req-a7c09a59-ec69-41a9-a904-fd95edc4d79b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 14:51:50.031506 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: warnings.warn(msg) May 25 14:51:50.031506 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: May 25 14:51:50.032381 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c09a59-ec69-41a9-a904-fd95edc4d79b admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:51:50.048333 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 745/1499] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:51:50.052607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a02c8-b2f6-418d-b1b7-7259ec70a40c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:51:50.062832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a02c8-b2f6-418d-b1b7-7259ec70a40c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:51:50.067517 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096c4f8-ef18-4874-9b3a-f3afd96c6eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:51:50.068181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096c4f8-ef18-4874-9b3a-f3afd96c6eec admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:51:50.068465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096c4f8-ef18-4874-9b3a-f3afd96c6eec admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:51:50.068738 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096c4f8-ef18-4874-9b3a-f3afd96c6eec admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:51:50.069450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c096c4f8-ef18-4874-9b3a-f3afd96c6eec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:51:50.070493 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: WARNING py.warnings [None req-c096c4f8-ef18-4874-9b3a-f3afd96c6eec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 14:51:50.070493 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: warnings.warn(msg) May 25 14:51:50.070493 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: May 25 14:51:50.071398 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c096c4f8-ef18-4874-9b3a-f3afd96c6eec admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:51:50.084621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 755/1500] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:51:50.088738 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c09a59-ec69-41a9-a904-fd95edc4d79b admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:51:50.099489 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c09a59-ec69-41a9-a904-fd95edc4d79b admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:51:50.104082 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798b4b1-e8d5-41cd-aa67-5a50ecc76a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:51:50.104952 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798b4b1-e8d5-41cd-aa67-5a50ecc76a30 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:51:50.105237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798b4b1-e8d5-41cd-aa67-5a50ecc76a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:51:50.105515 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798b4b1-e8d5-41cd-aa67-5a50ecc76a30 admin admin] PATH_INFO: `/v3/projects/ebf08db8ed09471ca711edc7c21ddcfe/users/5a5e8d723c594ce3844aaf4f371b579f/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:51:50.120710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798b4b1-e8d5-41cd-aa67-5a50ecc76a30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebf08db8ed09471ca711edc7c21ddcfe, user_id=5a5e8d723c594ce3844aaf4f371b579f, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:51:50.121745 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: WARNING py.warnings [None req-d798b4b1-e8d5-41cd-aa67-5a50ecc76a30 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:51:50.121745 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: warnings.warn(msg) May 25 14:51:50.121745 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: May 25 14:51:50.123719 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798b4b1-e8d5-41cd-aa67-5a50ecc76a30 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:51:50.140136 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 746/1501] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:51:50 2022] PUT /identity/v3/projects/ebf08db8ed09471ca711edc7c21ddcfe/users/5a5e8d723c594ce3844aaf4f371b579f/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:51:50.151542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb1703-352e-41ce-9678-f6eb1c882dee None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:51:50.151832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb1703-352e-41ce-9678-f6eb1c882dee None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:51:50.152114 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb1703-352e-41ce-9678-f6eb1c882dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:51:50.179110 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-00cb1703-352e-41ce-9678-f6eb1c882dee None None] MFA Rules not processed for user `5a5e8d723c594ce3844aaf4f371b579f`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:51:50.180311 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-00cb1703-352e-41ce-9678-f6eb1c882dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:51:50.244536 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 756/1502] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:51:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:51:50.257494 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0d986-5753-4ca6-b947-a64762edfa00 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:51:50.257783 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0d986-5753-4ca6-b947-a64762edfa00 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:51:50.258091 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0d986-5753-4ca6-b947-a64762edfa00 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:51:50.259482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 747/1503] 10.222.0.78 () {54 vars in 963 bytes} [Wed May 25 14:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:51:50.264431 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec52df9-003f-4036-a470-add729367ca1 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:51:50.264723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec52df9-003f-4036-a470-add729367ca1 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:51:50.265006 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec52df9-003f-4036-a470-add729367ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:51:50.411003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-6ec52df9-003f-4036-a470-add729367ca1 None None] MFA Rules not processed for user `50dbd3dd7e87481cb940a467c294006c`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:51:50.412219 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-6ec52df9-003f-4036-a470-add729367ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:51:50.460230 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:51:50.469484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 757/1504] 10.222.0.78 () {58 vars in 1060 bytes} [Wed May 25 14:51:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:51:50.477266 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0d986-5753-4ca6-b947-a64762edfa00 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:51:50.487831 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0d986-5753-4ca6-b947-a64762edfa00 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:51:50.491884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542aaef-e3c5-4555-b29d-d0bd13261d6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:51:50.492513 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542aaef-e3c5-4555-b29d-d0bd13261d6d service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:51:50.492788 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542aaef-e3c5-4555-b29d-d0bd13261d6d service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:51:50.493059 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542aaef-e3c5-4555-b29d-d0bd13261d6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:51:50.501566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1542aaef-e3c5-4555-b29d-d0bd13261d6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:51:50.502954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1542aaef-e3c5-4555-b29d-d0bd13261d6d service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:51:50.511670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 748/1505] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:51:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:51:50.553959 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcb7cd-75e3-42cb-be7c-08679f04d751 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:51:50.554290 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcb7cd-75e3-42cb-be7c-08679f04d751 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:51:50.554568 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcb7cd-75e3-42cb-be7c-08679f04d751 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:51:50.555894 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 758/1506] 10.222.0.78 () {52 vars in 829 bytes} [Wed May 25 14:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:51:50.559640 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542aaef-e3c5-4555-b29d-d0bd13261d6d service cinder] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:51:50.621126 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542aaef-e3c5-4555-b29d-d0bd13261d6d service cinder] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:51:50.626230 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02e641-57b0-442c-86b3-16c34d9457ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:51:50.627088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02e641-57b0-442c-86b3-16c34d9457ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:51:50.627365 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02e641-57b0-442c-86b3-16c34d9457ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:51:50.627634 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02e641-57b0-442c-86b3-16c34d9457ed admin admin] PATH_INFO: `/v3/users/5a5e8d723c594ce3844aaf4f371b579f` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:51:50.628984 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b02e641-57b0-442c-86b3-16c34d9457ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a5e8d723c594ce3844aaf4f371b579f)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:51:50.629997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: WARNING py.warnings [None req-2b02e641-57b0-442c-86b3-16c34d9457ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:51:50.629997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: warnings.warn(msg) May 25 14:51:50.629997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: May 25 14:51:50.631239 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b02e641-57b0-442c-86b3-16c34d9457ed admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:51:50.696629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2b02e641-57b0-442c-86b3-16c34d9457ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a5e8d723c594ce3844aaf4f371b579f'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:51:50.717211 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2b02e641-57b0-442c-86b3-16c34d9457ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a5e8d723c594ce3844aaf4f371b579f'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:51:50.726914 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2b02e641-57b0-442c-86b3-16c34d9457ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a5e8d723c594ce3844aaf4f371b579f'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:51:50.727736 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2b02e641-57b0-442c-86b3-16c34d9457ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a5e8d723c594ce3844aaf4f371b579f'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:51:50.728409 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2b02e641-57b0-442c-86b3-16c34d9457ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a5e8d723c594ce3844aaf4f371b579f'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:51:50.737721 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:51:50.748817 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 749/1507] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:51:50 2022] DELETE /identity/v3/users/5a5e8d723c594ce3844aaf4f371b579f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:51:50.753423 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02dc29b-f3ee-4466-b2f3-a2a43cde2ea7 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:51:50.753924 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02dc29b-f3ee-4466-b2f3-a2a43cde2ea7 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:51:50.753924 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02dc29b-f3ee-4466-b2f3-a2a43cde2ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:51:50.770908 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-d02dc29b-f3ee-4466-b2f3-a2a43cde2ea7 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:51:50.772105 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d02dc29b-f3ee-4466-b2f3-a2a43cde2ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:51:50.831132 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 759/1508] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:51:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:51:50.841278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02e641-57b0-442c-86b3-16c34d9457ed admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:51:50.842824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-2b02e641-57b0-442c-86b3-16c34d9457ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:51:50.901513 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02e641-57b0-442c-86b3-16c34d9457ed admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:51:50.905979 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37963a-af52-46c3-ab2d-fe4555e41835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:51:50.906780 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37963a-af52-46c3-ab2d-fe4555e41835 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:51:50.907068 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37963a-af52-46c3-ab2d-fe4555e41835 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:51:50.907353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37963a-af52-46c3-ab2d-fe4555e41835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:51:50.916761 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37963a-af52-46c3-ab2d-fe4555e41835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:51:50.918254 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37963a-af52-46c3-ab2d-fe4555e41835 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:51:50.928311 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 750/1509] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:51:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:51:51.286740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02dc29b-f3ee-4466-b2f3-a2a43cde2ea7 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:51:51.288244 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d02dc29b-f3ee-4466-b2f3-a2a43cde2ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:51:51.306845 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02dc29b-f3ee-4466-b2f3-a2a43cde2ea7 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:51:51.314347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03db927-d340-41e1-96cf-7a68cf2a24e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:51:51.315952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03db927-d340-41e1-96cf-7a68cf2a24e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:51:51.316526 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03db927-d340-41e1-96cf-7a68cf2a24e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:51:51.317118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03db927-d340-41e1-96cf-7a68cf2a24e4 admin admin] PATH_INFO: `/v3/projects/ebf08db8ed09471ca711edc7c21ddcfe` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:51:51.319470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03db927-d340-41e1-96cf-7a68cf2a24e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebf08db8ed09471ca711edc7c21ddcfe)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:51:51.321621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: WARNING py.warnings [None req-a03db927-d340-41e1-96cf-7a68cf2a24e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:51:51.321621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: warnings.warn(msg) May 25 14:51:51.321621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: May 25 14:51:51.324321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03db927-d340-41e1-96cf-7a68cf2a24e4 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:51:51.392183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a03db927-d340-41e1-96cf-7a68cf2a24e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebf08db8ed09471ca711edc7c21ddcfe', 'request_id': 'req-a03db927-d340-41e1-96cf-7a68cf2a24e4'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:51:51.398266 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a03db927-d340-41e1-96cf-7a68cf2a24e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebf08db8ed09471ca711edc7c21ddcfe', 'request_id': 'req-a03db927-d340-41e1-96cf-7a68cf2a24e4'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:51:51.414731 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a03db927-d340-41e1-96cf-7a68cf2a24e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebf08db8ed09471ca711edc7c21ddcfe', 'request_id': 'req-a03db927-d340-41e1-96cf-7a68cf2a24e4'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:51:51.431586 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a03db927-d340-41e1-96cf-7a68cf2a24e4 admin admin] The token cache is being invalidate because project ebf08db8ed09471ca711edc7c21ddcfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:51:51.431874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a03db927-d340-41e1-96cf-7a68cf2a24e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:51:51.432598 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a03db927-d340-41e1-96cf-7a68cf2a24e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:51:51.434842 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 760/1510] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:51:51 2022] DELETE /identity/v3/projects/ebf08db8ed09471ca711edc7c21ddcfe => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:52:00.924071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=29 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:52:01.428290 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:53:39.859097 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b08d7-5d6a-4e92-bf76-db0579424245 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:53:39.859796 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b08d7-5d6a-4e92-bf76-db0579424245 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:53:39.859796 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b08d7-5d6a-4e92-bf76-db0579424245 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:53:39.861019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:53:39.865928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983924b-1b2b-41dd-9c4b-16b0f8d6c611 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:53:39.866414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983924b-1b2b-41dd-9c4b-16b0f8d6c611 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:53:39.866679 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983924b-1b2b-41dd-9c4b-16b0f8d6c611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:53:39.885112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-3983924b-1b2b-41dd-9c4b-16b0f8d6c611 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:53:39.886365 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3983924b-1b2b-41dd-9c4b-16b0f8d6c611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:53:39.950980 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:53:40.135200 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b08d7-5d6a-4e92-bf76-db0579424245 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:53:40.137623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-fe9b08d7-5d6a-4e92-bf76-db0579424245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:53:40.215293 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b08d7-5d6a-4e92-bf76-db0579424245 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:53:40.221165 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697654d2-c75b-4d0f-b2ce-a338f754a2cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:53:40.221820 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697654d2-c75b-4d0f-b2ce-a338f754a2cd service nova] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:53:40.222132 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697654d2-c75b-4d0f-b2ce-a338f754a2cd service nova] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:53:40.222418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697654d2-c75b-4d0f-b2ce-a338f754a2cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:53:40.231002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697654d2-c75b-4d0f-b2ce-a338f754a2cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:53:40.232607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697654d2-c75b-4d0f-b2ce-a338f754a2cd service nova] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:53:40.242142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 752/1513] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:53:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:53:41.132377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23742629-28f4-4cd6-a3bd-6d025428bafa None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:53:41.132984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23742629-28f4-4cd6-a3bd-6d025428bafa None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:53:41.132984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23742629-28f4-4cd6-a3bd-6d025428bafa None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:53:41.134369 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:53:41.139428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11a94b-722d-4a86-b9ad-370380d44123 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:53:41.139867 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11a94b-722d-4a86-b9ad-370380d44123 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:53:41.139969 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11a94b-722d-4a86-b9ad-370380d44123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:53:41.158694 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-de11a94b-722d-4a86-b9ad-370380d44123 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:53:41.159818 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-de11a94b-722d-4a86-b9ad-370380d44123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:53:41.169117 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:53:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:53:41.323750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23742629-28f4-4cd6-a3bd-6d025428bafa None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:53:41.340811 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23742629-28f4-4cd6-a3bd-6d025428bafa None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:53:41.346565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1452c7-03ac-43c8-bc66-3da802c6b004 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:53:41.347217 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1452c7-03ac-43c8-bc66-3da802c6b004 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:53:41.347504 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1452c7-03ac-43c8-bc66-3da802c6b004 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:53:41.347792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1452c7-03ac-43c8-bc66-3da802c6b004 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:53:41.357097 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1452c7-03ac-43c8-bc66-3da802c6b004 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:53:41.358760 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1452c7-03ac-43c8-bc66-3da802c6b004 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:53:41.368617 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 763/1516] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:53:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:53:44.239076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff631e-be7f-4f1a-b03c-a14e090fc493 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:53:44.239723 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff631e-be7f-4f1a-b03c-a14e090fc493 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:53:44.239723 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff631e-be7f-4f1a-b03c-a14e090fc493 None None] PATH_INFO: `/v3` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:53:44.240982 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 754/1517] 10.222.0.72 () {54 vars in 925 bytes} [Wed May 25 14:53:44 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:53:44.255627 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeff41a-ee6f-4092-8296-033de49b9d36 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:53:44.256054 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeff41a-ee6f-4092-8296-033de49b9d36 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:53:44.256184 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeff41a-ee6f-4092-8296-033de49b9d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:53:44.276366 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-7aeff41a-ee6f-4092-8296-033de49b9d36 None None] MFA Rules not processed for user `63789c0a113447ff90933d447cf5bac3`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:53:44.277559 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7aeff41a-ee6f-4092-8296-033de49b9d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:53:44.339716 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 764/1518] 10.222.0.72 () {58 vars in 1015 bytes} [Wed May 25 14:53:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:53:51.153345 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=37 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:53:52.643753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0260c9-5edb-499b-8d39-9eb27bbf2c67 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:53:52.644433 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0260c9-5edb-499b-8d39-9eb27bbf2c67 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:53:52.644433 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0260c9-5edb-499b-8d39-9eb27bbf2c67 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:53:52.645732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 755/1519] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:53:52.651053 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e31d1-37dc-4806-a861-fcf3146fd563 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:53:52.652016 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e31d1-37dc-4806-a861-fcf3146fd563 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:53:52.652205 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e31d1-37dc-4806-a861-fcf3146fd563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:53:52.670105 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-f31e31d1-37dc-4806-a861-fcf3146fd563 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:53:52.671303 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f31e31d1-37dc-4806-a861-fcf3146fd563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:53:52.680155 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 765/1520] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:53:52.833927 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0260c9-5edb-499b-8d39-9eb27bbf2c67 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:53:52.850518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0260c9-5edb-499b-8d39-9eb27bbf2c67 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:53:52.856883 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d9ad2-a4d3-4bb3-b8a8-f35f0fe27325 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:53:52.857526 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d9ad2-a4d3-4bb3-b8a8-f35f0fe27325 service nova] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:53:52.857808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d9ad2-a4d3-4bb3-b8a8-f35f0fe27325 service nova] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:53:52.858115 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d9ad2-a4d3-4bb3-b8a8-f35f0fe27325 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:53:52.867041 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d9ad2-a4d3-4bb3-b8a8-f35f0fe27325 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:53:52.868967 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d9ad2-a4d3-4bb3-b8a8-f35f0fe27325 service nova] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:53:52.878568 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 756/1521] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:53:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:53:53.735913 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af959fa-99c9-4a0e-9c0e-14196516ba3e None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:53:53.736509 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af959fa-99c9-4a0e-9c0e-14196516ba3e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:53:53.736509 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af959fa-99c9-4a0e-9c0e-14196516ba3e None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:53:53.737848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:53:53.742784 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bced5b7-eec9-4107-b5aa-4df3e2ec3dea None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:53:53.743206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bced5b7-eec9-4107-b5aa-4df3e2ec3dea None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:53:53.743345 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bced5b7-eec9-4107-b5aa-4df3e2ec3dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:53:53.761946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-9bced5b7-eec9-4107-b5aa-4df3e2ec3dea None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:53:53.763167 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-9bced5b7-eec9-4107-b5aa-4df3e2ec3dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:53:53.770995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:53:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:53:53.923200 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af959fa-99c9-4a0e-9c0e-14196516ba3e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:53:53.937670 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af959fa-99c9-4a0e-9c0e-14196516ba3e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:53:53.943174 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f2913-5be2-491d-8a20-4576a89abaad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:53:53.943806 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f2913-5be2-491d-8a20-4576a89abaad service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:53:53.944092 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f2913-5be2-491d-8a20-4576a89abaad service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:53:53.944373 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f2913-5be2-491d-8a20-4576a89abaad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:53:53.953183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f2913-5be2-491d-8a20-4576a89abaad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:53:53.954835 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f2913-5be2-491d-8a20-4576a89abaad service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:53:53.964982 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 767/1524] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:53:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:53:56.306331 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70ee67-2961-4634-8a75-150a2b8cdb99 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:53:56.306867 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70ee67-2961-4634-8a75-150a2b8cdb99 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:53:56.306988 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70ee67-2961-4634-8a75-150a2b8cdb99 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:53:56.308853 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 758/1525] 10.222.0.78 () {54 vars in 960 bytes} [Wed May 25 14:53:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:53:56.313703 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be9dbc-a540-47fc-9e54-647b21f7ea9e None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:53:56.314173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be9dbc-a540-47fc-9e54-647b21f7ea9e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:53:56.314310 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be9dbc-a540-47fc-9e54-647b21f7ea9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:53:56.332885 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-07be9dbc-a540-47fc-9e54-647b21f7ea9e None None] MFA Rules not processed for user `63789c0a113447ff90933d447cf5bac3`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:53:56.334150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-07be9dbc-a540-47fc-9e54-647b21f7ea9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:53:56.342690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 768/1526] 10.222.0.78 () {58 vars in 1057 bytes} [Wed May 25 14:53:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:53:56.350131 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab70ee67-2961-4634-8a75-150a2b8cdb99 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:53:56.364057 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab70ee67-2961-4634-8a75-150a2b8cdb99 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:53:56.369020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22748871-92ff-452c-9e41-8b65c895e769 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63789c0a113447ff90933d447cf5bac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:53:56.369597 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22748871-92ff-452c-9e41-8b65c895e769 service placement] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:53:56.370503 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22748871-92ff-452c-9e41-8b65c895e769 service placement] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:53:56.370746 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22748871-92ff-452c-9e41-8b65c895e769 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:53:56.379141 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22748871-92ff-452c-9e41-8b65c895e769 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:53:56.380671 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22748871-92ff-452c-9e41-8b65c895e769 service placement] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:53:56.390463 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 759/1527] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:53:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:05.227353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d397e2-0424-407d-b199-f53ed71543f8 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:05.227927 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d397e2-0424-407d-b199-f53ed71543f8 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:05.227927 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d397e2-0424-407d-b199-f53ed71543f8 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:05.229251 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:54:05.234852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117af09b-7490-4db5-b7f0-335b3f3ca2b7 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:05.235336 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117af09b-7490-4db5-b7f0-335b3f3ca2b7 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:05.235445 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117af09b-7490-4db5-b7f0-335b3f3ca2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:05.254193 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-117af09b-7490-4db5-b7f0-335b3f3ca2b7 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:05.255421 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-117af09b-7490-4db5-b7f0-335b3f3ca2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:05.266782 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:54:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:05.423773 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d397e2-0424-407d-b199-f53ed71543f8 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:05.432587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=75 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:05.441050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d397e2-0424-407d-b199-f53ed71543f8 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:05.446574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8508d3-4ee3-4d3d-9e3b-34e99baac847 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:05.447238 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8508d3-4ee3-4d3d-9e3b-34e99baac847 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:05.447526 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8508d3-4ee3-4d3d-9e3b-34e99baac847 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:05.447807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8508d3-4ee3-4d3d-9e3b-34e99baac847 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:05.457896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8508d3-4ee3-4d3d-9e3b-34e99baac847 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:05.459579 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8508d3-4ee3-4d3d-9e3b-34e99baac847 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:05.471034 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 770/1530] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:54:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:06.360962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c78e34-8055-410a-b2e7-9183f79ea577 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:06.361544 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c78e34-8055-410a-b2e7-9183f79ea577 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:06.361544 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c78e34-8055-410a-b2e7-9183f79ea577 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:06.363031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:54:06.368441 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c80e75-e800-4a5b-8200-1c60a661fbbf None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:06.369015 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c80e75-e800-4a5b-8200-1c60a661fbbf None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:06.369015 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c80e75-e800-4a5b-8200-1c60a661fbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:06.390843 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-28c80e75-e800-4a5b-8200-1c60a661fbbf None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:06.392043 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-28c80e75-e800-4a5b-8200-1c60a661fbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:06.399989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:06.550722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c78e34-8055-410a-b2e7-9183f79ea577 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:06.565563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c78e34-8055-410a-b2e7-9183f79ea577 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:06.571047 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56273781-8eff-4941-b314-35318716a830 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:06.571691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56273781-8eff-4941-b314-35318716a830 service nova] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:06.571968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56273781-8eff-4941-b314-35318716a830 service nova] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:06.572236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56273781-8eff-4941-b314-35318716a830 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:06.580712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56273781-8eff-4941-b314-35318716a830 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:06.582336 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56273781-8eff-4941-b314-35318716a830 service nova] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:06.593156 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 762/1533] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:54:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:11.027344 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077cccc-00a3-440a-bcd9-b5688bc598f7 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:11.027908 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077cccc-00a3-440a-bcd9-b5688bc598f7 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:11.027908 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077cccc-00a3-440a-bcd9-b5688bc598f7 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:11.029306 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:54:11.035023 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca59300-a672-45f0-8729-6b286ca6852f None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:11.035485 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca59300-a672-45f0-8729-6b286ca6852f None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:11.035598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca59300-a672-45f0-8729-6b286ca6852f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:11.053922 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-aca59300-a672-45f0-8729-6b286ca6852f None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:11.055169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-aca59300-a672-45f0-8729-6b286ca6852f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:11.063830 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:11.100737 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077cccc-00a3-440a-bcd9-b5688bc598f7 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:11.102841 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c077cccc-00a3-440a-bcd9-b5688bc598f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:11.171848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077cccc-00a3-440a-bcd9-b5688bc598f7 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:11.175354 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52168d0f-5844-44ba-9cfe-2624e1163fd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:11.176008 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52168d0f-5844-44ba-9cfe-2624e1163fd9 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:11.176295 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52168d0f-5844-44ba-9cfe-2624e1163fd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:11.176571 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52168d0f-5844-44ba-9cfe-2624e1163fd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:11.187047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52168d0f-5844-44ba-9cfe-2624e1163fd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:11.188511 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52168d0f-5844-44ba-9cfe-2624e1163fd9 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:11.199373 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 773/1536] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:54:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:13.454948 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f332f-52f0-46c0-8152-1beef23431b5 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:13.455516 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f332f-52f0-46c0-8152-1beef23431b5 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:13.455516 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f332f-52f0-46c0-8152-1beef23431b5 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:13.456873 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:54:13.462170 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6df9f3-da2c-446a-8644-1e6f59bdd722 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:13.462592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6df9f3-da2c-446a-8644-1e6f59bdd722 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:13.462742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6df9f3-da2c-446a-8644-1e6f59bdd722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:13.480763 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-1f6df9f3-da2c-446a-8644-1e6f59bdd722 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:13.481981 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1f6df9f3-da2c-446a-8644-1e6f59bdd722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:13.490047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:13.526365 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f332f-52f0-46c0-8152-1beef23431b5 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:13.537417 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f332f-52f0-46c0-8152-1beef23431b5 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:13.541212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb92c0-9246-4b5f-aebc-e93ce63825e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:13.541884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb92c0-9246-4b5f-aebc-e93ce63825e2 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:13.542212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb92c0-9246-4b5f-aebc-e93ce63825e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:13.542447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb92c0-9246-4b5f-aebc-e93ce63825e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:13.553034 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb92c0-9246-4b5f-aebc-e93ce63825e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:13.554514 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb92c0-9246-4b5f-aebc-e93ce63825e2 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:13.566360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 765/1539] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:54:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:17.757751 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a61f0-30b4-46bf-9d3f-f6056218b1d9 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:17.758374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a61f0-30b4-46bf-9d3f-f6056218b1d9 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:17.758374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a61f0-30b4-46bf-9d3f-f6056218b1d9 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:17.759756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 775/1540] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:54:17.766056 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34baf694-c15d-4f84-9578-182f946c41ed None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:17.766645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34baf694-c15d-4f84-9578-182f946c41ed None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:17.766645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34baf694-c15d-4f84-9578-182f946c41ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:17.786639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-34baf694-c15d-4f84-9578-182f946c41ed None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:17.787986 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-34baf694-c15d-4f84-9578-182f946c41ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:17.797635 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 766/1541] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:18.079530 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a61f0-30b4-46bf-9d3f-f6056218b1d9 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:18.081546 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-671a61f0-30b4-46bf-9d3f-f6056218b1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:18.145749 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a61f0-30b4-46bf-9d3f-f6056218b1d9 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:18.154024 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d696d0-f53b-4ac2-9f60-dedccfc1af9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:18.155207 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d696d0-f53b-4ac2-9f60-dedccfc1af9a service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:18.155712 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d696d0-f53b-4ac2-9f60-dedccfc1af9a service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:18.156215 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d696d0-f53b-4ac2-9f60-dedccfc1af9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:18.173303 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d696d0-f53b-4ac2-9f60-dedccfc1af9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:18.176344 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d696d0-f53b-4ac2-9f60-dedccfc1af9a service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:18.195133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 776/1542] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:54:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:20.506560 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22311e-da0d-4c14-b098-962fe60c46bf None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:20.507196 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22311e-da0d-4c14-b098-962fe60c46bf None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:20.507309 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22311e-da0d-4c14-b098-962fe60c46bf None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:20.508680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 767/1543] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:54:20.514302 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a480a6-8c20-4af9-9864-74e1769a83a6 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:20.514818 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a480a6-8c20-4af9-9864-74e1769a83a6 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:20.514946 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a480a6-8c20-4af9-9864-74e1769a83a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:20.533729 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-19a480a6-8c20-4af9-9864-74e1769a83a6 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:54:20.534973 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-19a480a6-8c20-4af9-9864-74e1769a83a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:20.543515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 777/1544] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:54:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:54:20.822834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22311e-da0d-4c14-b098-962fe60c46bf None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:20.824549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-ed22311e-da0d-4c14-b098-962fe60c46bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:20.837753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22311e-da0d-4c14-b098-962fe60c46bf None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:20.841862 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0610c4b3-5d45-41b9-be71-aa1354509708 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:20.842547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610c4b3-5d45-41b9-be71-aa1354509708 service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:20.842827 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610c4b3-5d45-41b9-be71-aa1354509708 service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:20.843093 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610c4b3-5d45-41b9-be71-aa1354509708 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:20.852589 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0610c4b3-5d45-41b9-be71-aa1354509708 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:20.854714 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0610c4b3-5d45-41b9-be71-aa1354509708 service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:20.864569 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 768/1545] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:54:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:30.528061 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=66 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:30.860210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=42 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:56.519222 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827b9bd-3752-4fe8-8e33-9dd540f5f92a None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:56.519853 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827b9bd-3752-4fe8-8e33-9dd540f5f92a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:56.519853 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827b9bd-3752-4fe8-8e33-9dd540f5f92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:56.553521 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-4827b9bd-3752-4fe8-8e33-9dd540f5f92a None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:56.554787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4827b9bd-3752-4fe8-8e33-9dd540f5f92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:56.566205 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 778/1546] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:56.576838 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67299f3-2e6d-4c9e-84ea-c579838bdc4a None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:56.577822 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67299f3-2e6d-4c9e-84ea-c579838bdc4a None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:56.578336 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67299f3-2e6d-4c9e-84ea-c579838bdc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:56.614442 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-f67299f3-2e6d-4c9e-84ea-c579838bdc4a None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:56.616137 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f67299f3-2e6d-4c9e-84ea-c579838bdc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:56.626944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 769/1547] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:56.632851 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827b9bd-3752-4fe8-8e33-9dd540f5f92a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:56.648235 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827b9bd-3752-4fe8-8e33-9dd540f5f92a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:56.657385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dbe914-3bae-48c3-aa0d-3bb771f043ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:56.658423 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbe914-3bae-48c3-aa0d-3bb771f043ba admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:56.658974 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbe914-3bae-48c3-aa0d-3bb771f043ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:56.659428 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbe914-3bae-48c3-aa0d-3bb771f043ba admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:56.660344 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dbe914-3bae-48c3-aa0d-3bb771f043ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:55:56.660829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dbe914-3bae-48c3-aa0d-3bb771f043ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:56.662676 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dbe914-3bae-48c3-aa0d-3bb771f043ba admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:56.683194 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 779/1548] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:55:56.692255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67299f3-2e6d-4c9e-84ea-c579838bdc4a None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:56.708484 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67299f3-2e6d-4c9e-84ea-c579838bdc4a None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:56.719792 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5894f2d-1694-4660-adac-2e892d9d8ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:56.720535 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5894f2d-1694-4660-adac-2e892d9d8ba6 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:56.720815 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5894f2d-1694-4660-adac-2e892d9d8ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:56.721088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5894f2d-1694-4660-adac-2e892d9d8ba6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:56.721796 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5894f2d-1694-4660-adac-2e892d9d8ba6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:56.723932 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5894f2d-1694-4660-adac-2e892d9d8ba6 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:56.773199 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 770/1549] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:55:56 2022] POST /identity/v3/projects => generated 375 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:56.777761 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dbe914-3bae-48c3-aa0d-3bb771f043ba admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:56.792002 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dbe914-3bae-48c3-aa0d-3bb771f043ba admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:56.859686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f78425-cd76-4d77-92f4-1b4518f01766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:56.860840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f78425-cd76-4d77-92f4-1b4518f01766 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:56.861326 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f78425-cd76-4d77-92f4-1b4518f01766 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:56.861803 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f78425-cd76-4d77-92f4-1b4518f01766 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:56.862745 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f78425-cd76-4d77-92f4-1b4518f01766 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:56.863068 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb29de2-0017-4ad7-9540-0de5646017c5 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:56.863445 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb29de2-0017-4ad7-9540-0de5646017c5 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:56.863931 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb29de2-0017-4ad7-9540-0de5646017c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:56.865556 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f78425-cd76-4d77-92f4-1b4518f01766 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:56.896544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 780/1550] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:55:56 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:56.897905 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-2fb29de2-0017-4ad7-9540-0de5646017c5 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:56.902203 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-2fb29de2-0017-4ad7-9540-0de5646017c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:56.903126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5f912-89b2-4640-a728-0106647b3a1a None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:56.903621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5f912-89b2-4640-a728-0106647b3a1a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:56.904099 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5f912-89b2-4640-a728-0106647b3a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:56.923515 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 771/1551] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:56.925720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb29de2-0017-4ad7-9540-0de5646017c5 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:56.937250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-02e5f912-89b2-4640-a728-0106647b3a1a None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:56.938647 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-02e5f912-89b2-4640-a728-0106647b3a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:56.955290 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 781/1552] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:56.956778 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbecf60c-0a6d-4be2-a2cc-931a18c5a2af None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:56.957281 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbecf60c-0a6d-4be2-a2cc-931a18c5a2af None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:56.957644 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbecf60c-0a6d-4be2-a2cc-931a18c5a2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:56.959974 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb29de2-0017-4ad7-9540-0de5646017c5 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:56.973304 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6872676-8239-46ee-8ce9-f392db9b166b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:56.974120 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6872676-8239-46ee-8ce9-f392db9b166b admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:56.974531 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6872676-8239-46ee-8ce9-f392db9b166b admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:56.974864 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6872676-8239-46ee-8ce9-f392db9b166b admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:56.976172 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6872676-8239-46ee-8ce9-f392db9b166b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:56.977852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6872676-8239-46ee-8ce9-f392db9b166b admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:56.983349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-bbecf60c-0a6d-4be2-a2cc-931a18c5a2af None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:56.984641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-bbecf60c-0a6d-4be2-a2cc-931a18c5a2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:56.996817 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 772/1553] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:57.002297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e560a5-585b-44c2-b7f5-467dd72afac4 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.002588 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e560a5-585b-44c2-b7f5-467dd72afac4 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.002709 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 782/1554] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:57.002881 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e560a5-585b-44c2-b7f5-467dd72afac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.003998 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f508a8-6b85-4475-ac66-69ffb48dc0d6 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.004398 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f508a8-6b85-4475-ac66-69ffb48dc0d6 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.004679 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f508a8-6b85-4475-ac66-69ffb48dc0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.025947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-21f508a8-6b85-4475-ac66-69ffb48dc0d6 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:57.027545 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-21f508a8-6b85-4475-ac66-69ffb48dc0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:57.030864 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-05e560a5-585b-44c2-b7f5-467dd72afac4 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:57.032052 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-05e560a5-585b-44c2-b7f5-467dd72afac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:57.037231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 783/1555] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:55:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:57.039030 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b195bd2-ea7c-45ab-8bf1-ce3173b134fc None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.039554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b195bd2-ea7c-45ab-8bf1-ce3173b134fc None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.040052 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b195bd2-ea7c-45ab-8bf1-ce3173b134fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.041748 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 773/1556] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:57.042654 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e560a5-585b-44c2-b7f5-467dd72afac4 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:57.063587 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e560a5-585b-44c2-b7f5-467dd72afac4 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:57.066278 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-4b195bd2-ea7c-45ab-8bf1-ce3173b134fc None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:57.067563 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4b195bd2-ea7c-45ab-8bf1-ce3173b134fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:57.068428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70300ad1-771a-45a4-882a-d3b4abbecd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:57.069099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70300ad1-771a-45a4-882a-d3b4abbecd76 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.069376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70300ad1-771a-45a4-882a-d3b4abbecd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.069647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70300ad1-771a-45a4-882a-d3b4abbecd76 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.071024 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70300ad1-771a-45a4-882a-d3b4abbecd76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:57.072691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70300ad1-771a-45a4-882a-d3b4abbecd76 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:57.076721 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 784/1557] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:55:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:57.078131 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d5b4d-b08b-4a99-9ad2-0ee99457018d None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.078834 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d5b4d-b08b-4a99-9ad2-0ee99457018d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.078834 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d5b4d-b08b-4a99-9ad2-0ee99457018d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.084890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 774/1558] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:57.086104 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70300ad1-771a-45a4-882a-d3b4abbecd76 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:57.095948 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70300ad1-771a-45a4-882a-d3b4abbecd76 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:57.097223 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-ac5d5b4d-b08b-4a99-9ad2-0ee99457018d None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:57.098456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ac5d5b4d-b08b-4a99-9ad2-0ee99457018d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:57.100537 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675cfcc4-c2b8-484c-a6d7-08732d161641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:57.101300 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cfcc4-c2b8-484c-a6d7-08732d161641 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.101578 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cfcc4-c2b8-484c-a6d7-08732d161641 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.101845 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cfcc4-c2b8-484c-a6d7-08732d161641 admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.102594 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675cfcc4-c2b8-484c-a6d7-08732d161641 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:55:57.102896 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675cfcc4-c2b8-484c-a6d7-08732d161641 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:57.103894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: WARNING py.warnings [None req-675cfcc4-c2b8-484c-a6d7-08732d161641 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 14:55:57.103894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: warnings.warn(msg) May 25 14:55:57.103894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: May 25 14:55:57.104766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675cfcc4-c2b8-484c-a6d7-08732d161641 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:57.106336 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 785/1559] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:55:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:57.107110 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d5b4d-b08b-4a99-9ad2-0ee99457018d None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:57.116969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d5b4d-b08b-4a99-9ad2-0ee99457018d None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:57.122417 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a90ce7-b86c-4d8a-8ad0-968ada972293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:57.123276 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a90ce7-b86c-4d8a-8ad0-968ada972293 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.124195 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a90ce7-b86c-4d8a-8ad0-968ada972293 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.124477 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a90ce7-b86c-4d8a-8ad0-968ada972293 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.125216 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a90ce7-b86c-4d8a-8ad0-968ada972293 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:55:57.125401 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 775/1560] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:55:57.125566 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a90ce7-b86c-4d8a-8ad0-968ada972293 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:57.126672 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d86d38-41b6-4e21-857e-68144ef1940b None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.126958 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d86d38-41b6-4e21-857e-68144ef1940b None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.127228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a90ce7-b86c-4d8a-8ad0-968ada972293 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:57.127338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d86d38-41b6-4e21-857e-68144ef1940b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.141164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 786/1561] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:55:57.142361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fc8d1-4f42-42e1-988c-0d4e7b38dddc None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.142701 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fc8d1-4f42-42e1-988c-0d4e7b38dddc None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.142939 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fc8d1-4f42-42e1-988c-0d4e7b38dddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.148978 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-d6d86d38-41b6-4e21-857e-68144ef1940b None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:57.150209 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d6d86d38-41b6-4e21-857e-68144ef1940b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:57.159717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 776/1562] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:55:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:57.161559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e26ea5-8aec-433f-b496-6c7fcc3d126c None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.162173 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e26ea5-8aec-433f-b496-6c7fcc3d126c None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.162462 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e26ea5-8aec-433f-b496-6c7fcc3d126c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.166062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-067fc8d1-4f42-42e1-988c-0d4e7b38dddc None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:57.167275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-067fc8d1-4f42-42e1-988c-0d4e7b38dddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:57.175399 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 787/1563] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:55:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:57.176760 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62899628-89b0-4778-afe7-9df0398a67ba None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.177054 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62899628-89b0-4778-afe7-9df0398a67ba None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.177333 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62899628-89b0-4778-afe7-9df0398a67ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.180139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-d8e26ea5-8aec-433f-b496-6c7fcc3d126c None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:57.181329 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d8e26ea5-8aec-433f-b496-6c7fcc3d126c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:57.190467 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 777/1564] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:55:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:57.191719 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e26ea5-8aec-433f-b496-6c7fcc3d126c None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:57.196499 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-62899628-89b0-4778-afe7-9df0398a67ba None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:57.197707 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-62899628-89b0-4778-afe7-9df0398a67ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:57.202608 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e26ea5-8aec-433f-b496-6c7fcc3d126c None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:57.206518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 788/1565] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:55:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:57.207842 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a433b7d-26a7-40cd-83c7-f2a596f4f2cc None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.208137 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a433b7d-26a7-40cd-83c7-f2a596f4f2cc None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.208990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a433b7d-26a7-40cd-83c7-f2a596f4f2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.210518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abac77-d177-4147-80f2-5c8cffe998c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:57.211472 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abac77-d177-4147-80f2-5c8cffe998c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.211757 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abac77-d177-4147-80f2-5c8cffe998c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.212033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abac77-d177-4147-80f2-5c8cffe998c5 admin admin] PATH_INFO: `/v3/projects/2395f31fad3645c19096d648c521bf31/users/7fedb9b92e9a4e3cb671f69751e426a1/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.227523 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-3a433b7d-26a7-40cd-83c7-f2a596f4f2cc None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:57.227737 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6abac77-d177-4147-80f2-5c8cffe998c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2395f31fad3645c19096d648c521bf31, user_id=7fedb9b92e9a4e3cb671f69751e426a1, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:57.228716 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3a433b7d-26a7-40cd-83c7-f2a596f4f2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:57.230404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6abac77-d177-4147-80f2-5c8cffe998c5 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:57.237725 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 789/1566] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:55:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:57.238545 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a433b7d-26a7-40cd-83c7-f2a596f4f2cc None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:57.247282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 778/1567] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:55:57 2022] PUT /identity/v3/projects/2395f31fad3645c19096d648c521bf31/users/7fedb9b92e9a4e3cb671f69751e426a1/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:57.248003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a433b7d-26a7-40cd-83c7-f2a596f4f2cc None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:57.248130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abac77-d177-4147-80f2-5c8cffe998c5 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:57.309669 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1169da7-d8e0-4c44-8d0e-493a63ee217b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:57.310366 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1169da7-d8e0-4c44-8d0e-493a63ee217b admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.310674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1169da7-d8e0-4c44-8d0e-493a63ee217b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.311530 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1169da7-d8e0-4c44-8d0e-493a63ee217b admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.312226 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1169da7-d8e0-4c44-8d0e-493a63ee217b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:57.313377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: WARNING py.warnings [None req-b1169da7-d8e0-4c44-8d0e-493a63ee217b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:55:57.313377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: warnings.warn(msg) May 25 14:55:57.313377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: May 25 14:55:57.314320 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abac77-d177-4147-80f2-5c8cffe998c5 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:57.314618 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1169da7-d8e0-4c44-8d0e-493a63ee217b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:57.319337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab612fe-f31b-402e-9bb7-10535155047c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:57.319994 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab612fe-f31b-402e-9bb7-10535155047c admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.320277 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab612fe-f31b-402e-9bb7-10535155047c admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.320554 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab612fe-f31b-402e-9bb7-10535155047c admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.321244 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab612fe-f31b-402e-9bb7-10535155047c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:57.322974 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:57.323237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab612fe-f31b-402e-9bb7-10535155047c admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:57.344487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 790/1568] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:55:57 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:57.345396 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1169da7-d8e0-4c44-8d0e-493a63ee217b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:57.363340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 779/1569] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:55:57 2022] POST /identity/v3/projects => generated 367 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:57.364864 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a3e6c-7e7b-464e-af8e-8324a0756437 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.365153 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a3e6c-7e7b-464e-af8e-8324a0756437 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.365433 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a3e6c-7e7b-464e-af8e-8324a0756437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.385275 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-8c3a3e6c-7e7b-464e-af8e-8324a0756437 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:57.386546 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-8c3a3e6c-7e7b-464e-af8e-8324a0756437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:57.413631 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:57.417341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1169da7-d8e0-4c44-8d0e-493a63ee217b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:57.448328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 780/1570] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:55:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:57.450057 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a12306-b82d-4e67-b2e8-6b227c5c2214 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.450744 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a12306-b82d-4e67-b2e8-6b227c5c2214 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.451026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a12306-b82d-4e67-b2e8-6b227c5c2214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.469115 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-30a12306-b82d-4e67-b2e8-6b227c5c2214 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:57.470330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-30a12306-b82d-4e67-b2e8-6b227c5c2214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:57.474236 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912d72e-1110-4cb2-8e9c-9c338cb407b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:57.474932 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912d72e-1110-4cb2-8e9c-9c338cb407b9 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.475218 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912d72e-1110-4cb2-8e9c-9c338cb407b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.475489 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912d72e-1110-4cb2-8e9c-9c338cb407b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.476204 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912d72e-1110-4cb2-8e9c-9c338cb407b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:55:57.476508 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912d72e-1110-4cb2-8e9c-9c338cb407b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:57.478157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912d72e-1110-4cb2-8e9c-9c338cb407b9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:57.478292 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 781/1571] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:55:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:57.479121 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a12306-b82d-4e67-b2e8-6b227c5c2214 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:57.490594 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 791/1572] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:55:57.491485 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912d72e-1110-4cb2-8e9c-9c338cb407b9 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:57.492297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a12306-b82d-4e67-b2e8-6b227c5c2214 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:57.497050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2390dd4-00b4-4dfc-b671-8b34efe0abaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:57.498049 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2390dd4-00b4-4dfc-b671-8b34efe0abaa admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.498049 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2390dd4-00b4-4dfc-b671-8b34efe0abaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.498984 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2390dd4-00b4-4dfc-b671-8b34efe0abaa admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.499538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2390dd4-00b4-4dfc-b671-8b34efe0abaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:55:57.499842 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2390dd4-00b4-4dfc-b671-8b34efe0abaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:57.501462 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2390dd4-00b4-4dfc-b671-8b34efe0abaa admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:57.502569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912d72e-1110-4cb2-8e9c-9c338cb407b9 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:57.507143 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a055c5f-8452-4990-81ae-3ebc90a301a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:57.507792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a055c5f-8452-4990-81ae-3ebc90a301a3 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.508075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a055c5f-8452-4990-81ae-3ebc90a301a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.508379 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a055c5f-8452-4990-81ae-3ebc90a301a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.509114 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a055c5f-8452-4990-81ae-3ebc90a301a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:55:57.509419 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a055c5f-8452-4990-81ae-3ebc90a301a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:57.511044 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a055c5f-8452-4990-81ae-3ebc90a301a3 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:57.513386 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 782/1573] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:55:57.515245 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205e936-06e4-4137-9294-c68baccdeece None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.515536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205e936-06e4-4137-9294-c68baccdeece None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.515816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205e936-06e4-4137-9294-c68baccdeece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.531820 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 792/1574] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:55:57.531820 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a055c5f-8452-4990-81ae-3ebc90a301a3 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:57.538708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-b205e936-06e4-4137-9294-c68baccdeece None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:57.539602 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a055c5f-8452-4990-81ae-3ebc90a301a3 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:57.540104 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-b205e936-06e4-4137-9294-c68baccdeece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:57.544536 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ed9b9-fd65-4244-8870-573bdcfe0538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:57.545252 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ed9b9-fd65-4244-8870-573bdcfe0538 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.545542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ed9b9-fd65-4244-8870-573bdcfe0538 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.545823 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ed9b9-fd65-4244-8870-573bdcfe0538 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.546551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ed9b9-fd65-4244-8870-573bdcfe0538 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:57.548131 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 783/1575] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:55:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:57.548482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ed9b9-fd65-4244-8870-573bdcfe0538 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:57.550825 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205e936-06e4-4137-9294-c68baccdeece None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:57.562217 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205e936-06e4-4137-9294-c68baccdeece None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:57.567501 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772a001-c804-4314-b289-c3038c770def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:57.568170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772a001-c804-4314-b289-c3038c770def admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.568450 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772a001-c804-4314-b289-c3038c770def admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.568720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772a001-c804-4314-b289-c3038c770def admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.569410 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6772a001-c804-4314-b289-c3038c770def admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:57.570483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: WARNING py.warnings [None req-6772a001-c804-4314-b289-c3038c770def admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:55:57.570483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: warnings.warn(msg) May 25 14:55:57.570483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: May 25 14:55:57.573974 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 793/1576] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:55:57 2022] POST /identity/v3/users => generated 292 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:57.573974 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ed9b9-fd65-4244-8870-573bdcfe0538 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:57.574203 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6772a001-c804-4314-b289-c3038c770def admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:57.585406 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ed9b9-fd65-4244-8870-573bdcfe0538 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:57.593369 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 784/1577] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:55:57 2022] POST /identity/v3/users => generated 284 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:57.594180 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772a001-c804-4314-b289-c3038c770def admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:57.595365 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cf5d9-27a5-44ac-949a-8801828f4155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:57.596042 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cf5d9-27a5-44ac-949a-8801828f4155 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.596329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cf5d9-27a5-44ac-949a-8801828f4155 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.596752 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cf5d9-27a5-44ac-949a-8801828f4155 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.597341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279cf5d9-27a5-44ac-949a-8801828f4155 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:55:57.597655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279cf5d9-27a5-44ac-949a-8801828f4155 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:57.599326 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279cf5d9-27a5-44ac-949a-8801828f4155 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:57.604680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772a001-c804-4314-b289-c3038c770def admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:57.610005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8261b2-e0a4-41bc-b23c-7d94ae3e0094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:57.610749 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8261b2-e0a4-41bc-b23c-7d94ae3e0094 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.611072 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8261b2-e0a4-41bc-b23c-7d94ae3e0094 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.611342 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8261b2-e0a4-41bc-b23c-7d94ae3e0094 admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.612067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8261b2-e0a4-41bc-b23c-7d94ae3e0094 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:55:57.612364 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8261b2-e0a4-41bc-b23c-7d94ae3e0094 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:57.613793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 794/1578] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:55:57.613947 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8261b2-e0a4-41bc-b23c-7d94ae3e0094 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:57.614629 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cf5d9-27a5-44ac-949a-8801828f4155 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:57.630197 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 785/1579] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:55:57.630412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cf5d9-27a5-44ac-949a-8801828f4155 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:57.630512 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8261b2-e0a4-41bc-b23c-7d94ae3e0094 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:57.634457 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24126dd-9e90-4499-8f6d-a9ec701c2853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:57.635154 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24126dd-9e90-4499-8f6d-a9ec701c2853 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.635440 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24126dd-9e90-4499-8f6d-a9ec701c2853 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.635720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24126dd-9e90-4499-8f6d-a9ec701c2853 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.636406 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24126dd-9e90-4499-8f6d-a9ec701c2853 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:57.638385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24126dd-9e90-4499-8f6d-a9ec701c2853 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:57.644793 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8261b2-e0a4-41bc-b23c-7d94ae3e0094 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:57.650144 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb91a68-2e48-4332-823d-5208fb4e0c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:57.650949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb91a68-2e48-4332-823d-5208fb4e0c4a admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.651228 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb91a68-2e48-4332-823d-5208fb4e0c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.651499 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb91a68-2e48-4332-823d-5208fb4e0c4a admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.652182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb91a68-2e48-4332-823d-5208fb4e0c4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:57.654127 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb91a68-2e48-4332-823d-5208fb4e0c4a admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:57.668274 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 795/1580] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:55:57 2022] POST /identity/v3/projects => generated 393 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:57.668274 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24126dd-9e90-4499-8f6d-a9ec701c2853 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:57.684505 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 786/1581] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:55:57 2022] POST /identity/v3/projects => generated 405 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:57.684505 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb91a68-2e48-4332-823d-5208fb4e0c4a admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:57.699423 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:57.737400 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24126dd-9e90-4499-8f6d-a9ec701c2853 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:57.738304 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb91a68-2e48-4332-823d-5208fb4e0c4a admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:57.742105 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a8fdc-062d-4186-992d-4595e937994d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:57.742767 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a8fdc-062d-4186-992d-4595e937994d service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.743046 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a8fdc-062d-4186-992d-4595e937994d service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.743345 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a8fdc-062d-4186-992d-4595e937994d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.795111 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b306f14-caad-4fa3-8fb7-5bf91607de3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:57.795785 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b306f14-caad-4fa3-8fb7-5bf91607de3b admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.796066 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b306f14-caad-4fa3-8fb7-5bf91607de3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.796346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b306f14-caad-4fa3-8fb7-5bf91607de3b admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.797034 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b306f14-caad-4fa3-8fb7-5bf91607de3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:57.799010 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b306f14-caad-4fa3-8fb7-5bf91607de3b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:57.805266 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a8fdc-062d-4186-992d-4595e937994d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:57.807061 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a8fdc-062d-4186-992d-4595e937994d service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:57.818318 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 787/1582] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:55:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:57.819079 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a8fdc-062d-4186-992d-4595e937994d service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:57.823779 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:57.828602 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 796/1583] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:55:57 2022] POST /identity/v3/projects => generated 357 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:57.828602 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b306f14-caad-4fa3-8fb7-5bf91607de3b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:57.829818 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a8fdc-062d-4186-992d-4595e937994d service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:57.895541 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2646515-6673-4976-8920-14adb9108c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:57.896940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2646515-6673-4976-8920-14adb9108c44 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.897228 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2646515-6673-4976-8920-14adb9108c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.897503 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2646515-6673-4976-8920-14adb9108c44 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.898337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2646515-6673-4976-8920-14adb9108c44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:57.899900 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2646515-6673-4976-8920-14adb9108c44 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:57.908820 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b306f14-caad-4fa3-8fb7-5bf91607de3b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:57.914069 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 788/1584] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:57.917723 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2646515-6673-4976-8920-14adb9108c44 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:57.921318 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0616067-e6d9-4cec-b822-d8fc30cabc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:57.921318 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0616067-e6d9-4cec-b822-d8fc30cabc84 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.921318 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0616067-e6d9-4cec-b822-d8fc30cabc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.921318 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0616067-e6d9-4cec-b822-d8fc30cabc84 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.922115 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0616067-e6d9-4cec-b822-d8fc30cabc84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:57.924077 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0616067-e6d9-4cec-b822-d8fc30cabc84 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:57.934887 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2646515-6673-4976-8920-14adb9108c44 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:57.940766 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 797/1585] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:57.943203 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf27265-f3fd-42bd-bad4-e328f9bc95f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:57.943991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf27265-f3fd-42bd-bad4-e328f9bc95f7 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.944328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf27265-f3fd-42bd-bad4-e328f9bc95f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.944615 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf27265-f3fd-42bd-bad4-e328f9bc95f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.945035 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0616067-e6d9-4cec-b822-d8fc30cabc84 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:57.945460 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf27265-f3fd-42bd-bad4-e328f9bc95f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:57.947517 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf27265-f3fd-42bd-bad4-e328f9bc95f7 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:57.955873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0616067-e6d9-4cec-b822-d8fc30cabc84 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:57.961709 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e654f0-693f-441a-9a5b-730a88ae9ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:57.962415 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e654f0-693f-441a-9a5b-730a88ae9ba7 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:57.962849 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e654f0-693f-441a-9a5b-730a88ae9ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:57.962985 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e654f0-693f-441a-9a5b-730a88ae9ba7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:57.964372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e654f0-693f-441a-9a5b-730a88ae9ba7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:57.966606 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e654f0-693f-441a-9a5b-730a88ae9ba7 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:57.981355 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 789/1586] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:55:57 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:57.981355 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf27265-f3fd-42bd-bad4-e328f9bc95f7 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.005286 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 798/1587] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:55:57 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:58.005286 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e654f0-693f-441a-9a5b-730a88ae9ba7 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.051872 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:58.064377 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf27265-f3fd-42bd-bad4-e328f9bc95f7 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.098772 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e654f0-693f-441a-9a5b-730a88ae9ba7 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.108325 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042668d-790b-469c-be8a-a8829dae4adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.109094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042668d-790b-469c-be8a-a8829dae4adc admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.109443 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042668d-790b-469c-be8a-a8829dae4adc admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.109803 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042668d-790b-469c-be8a-a8829dae4adc admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.110607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3042668d-790b-469c-be8a-a8829dae4adc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.112847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3042668d-790b-469c-be8a-a8829dae4adc admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.132586 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a29df-8b6e-4743-b966-635c471a3ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.133491 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a29df-8b6e-4743-b966-635c471a3ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.133964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a29df-8b6e-4743-b966-635c471a3ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.134329 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a29df-8b6e-4743-b966-635c471a3ea7 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.135204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a29df-8b6e-4743-b966-635c471a3ea7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.137280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a29df-8b6e-4743-b966-635c471a3ea7 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.143137 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 799/1588] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:55:58 2022] POST /identity/v3/users => generated 303 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:58.143137 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042668d-790b-469c-be8a-a8829dae4adc admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.156886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042668d-790b-469c-be8a-a8829dae4adc admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.161745 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be6132-e3b1-4ac6-827c-c037fb9b6ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.162464 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 790/1589] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:55:57 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:58.162937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be6132-e3b1-4ac6-827c-c037fb9b6ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.163224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be6132-e3b1-4ac6-827c-c037fb9b6ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.163501 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be6132-e3b1-4ac6-827c-c037fb9b6ca5 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.164156 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a29df-8b6e-4743-b966-635c471a3ea7 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.164280 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be6132-e3b1-4ac6-827c-c037fb9b6ca5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.166369 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be6132-e3b1-4ac6-827c-c037fb9b6ca5 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.178357 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a29df-8b6e-4743-b966-635c471a3ea7 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.184393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4002d79e-3bec-4e35-8b23-867b4533856f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.185156 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002d79e-3bec-4e35-8b23-867b4533856f admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.185442 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002d79e-3bec-4e35-8b23-867b4533856f admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.186637 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002d79e-3bec-4e35-8b23-867b4533856f admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.187464 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4002d79e-3bec-4e35-8b23-867b4533856f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.189215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4002d79e-3bec-4e35-8b23-867b4533856f admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.195280 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 800/1590] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:55:58 2022] POST /identity/v3/users => generated 279 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:58.195280 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be6132-e3b1-4ac6-827c-c037fb9b6ca5 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.201346 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 791/1591] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:58.206220 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4002d79e-3bec-4e35-8b23-867b4533856f admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.206680 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be6132-e3b1-4ac6-827c-c037fb9b6ca5 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.211619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3907deb-1f06-4202-b054-5633288bd869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.212394 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3907deb-1f06-4202-b054-5633288bd869 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.212705 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3907deb-1f06-4202-b054-5633288bd869 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.212951 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:58.213098 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3907deb-1f06-4202-b054-5633288bd869 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.214047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3907deb-1f06-4202-b054-5633288bd869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.215796 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3907deb-1f06-4202-b054-5633288bd869 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.219920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4002d79e-3bec-4e35-8b23-867b4533856f admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.229953 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9061d9-9daf-4c0f-a081-e932f242faf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.230654 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9061d9-9daf-4c0f-a081-e932f242faf1 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.230927 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9061d9-9daf-4c0f-a081-e932f242faf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.231201 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9061d9-9daf-4c0f-a081-e932f242faf1 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.234920 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 801/1592] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:58.235090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9061d9-9daf-4c0f-a081-e932f242faf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.235190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3907deb-1f06-4202-b054-5633288bd869 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.239640 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9061d9-9daf-4c0f-a081-e932f242faf1 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.256946 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3907deb-1f06-4202-b054-5633288bd869 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.265666 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38d0aa-00de-43c5-a97b-4e952621f66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.266780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38d0aa-00de-43c5-a97b-4e952621f66a admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.266896 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 792/1593] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:55:58 2022] POST /identity/v3/users => generated 287 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:58.267356 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38d0aa-00de-43c5-a97b-4e952621f66a admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.267792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38d0aa-00de-43c5-a97b-4e952621f66a admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.267898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9061d9-9daf-4c0f-a081-e932f242faf1 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.268809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd38d0aa-00de-43c5-a97b-4e952621f66a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.271591 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd38d0aa-00de-43c5-a97b-4e952621f66a admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.281465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9061d9-9daf-4c0f-a081-e932f242faf1 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.287161 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be36ab-7aa5-4993-b9ab-266929e751ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.287943 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be36ab-7aa5-4993-b9ab-266929e751ab admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.288252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be36ab-7aa5-4993-b9ab-266929e751ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.288515 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be36ab-7aa5-4993-b9ab-266929e751ab admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.289471 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be36ab-7aa5-4993-b9ab-266929e751ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.291197 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be36ab-7aa5-4993-b9ab-266929e751ab admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.301102 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 802/1594] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:55:58 2022] POST /identity/v3/users => generated 289 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:58.302223 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38d0aa-00de-43c5-a97b-4e952621f66a admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.302872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 793/1595] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:58.303695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be36ab-7aa5-4993-b9ab-266929e751ab admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.316301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38d0aa-00de-43c5-a97b-4e952621f66a admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.323176 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be36ab-7aa5-4993-b9ab-266929e751ab admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.325302 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794634b1-e598-439f-b41e-a71dc93a86ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.326187 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794634b1-e598-439f-b41e-a71dc93a86ed admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.327898 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794634b1-e598-439f-b41e-a71dc93a86ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.328334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794634b1-e598-439f-b41e-a71dc93a86ed admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.329273 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794634b1-e598-439f-b41e-a71dc93a86ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.329402 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f807e92-c213-4108-a9a3-4373178e23b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.330223 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f807e92-c213-4108-a9a3-4373178e23b9 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.330550 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f807e92-c213-4108-a9a3-4373178e23b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.330904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f807e92-c213-4108-a9a3-4373178e23b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.331901 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f807e92-c213-4108-a9a3-4373178e23b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.333116 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794634b1-e598-439f-b41e-a71dc93a86ed admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.334435 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f807e92-c213-4108-a9a3-4373178e23b9 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.349270 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 794/1596] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:58.349270 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f807e92-c213-4108-a9a3-4373178e23b9 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.351873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 803/1597] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:58.355502 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794634b1-e598-439f-b41e-a71dc93a86ed admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.361414 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f807e92-c213-4108-a9a3-4373178e23b9 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.366428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f40f88-0b56-4cd8-b5a2-153b551fdc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.367314 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f40f88-0b56-4cd8-b5a2-153b551fdc25 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.367586 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f40f88-0b56-4cd8-b5a2-153b551fdc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.367854 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f40f88-0b56-4cd8-b5a2-153b551fdc25 admin admin] PATH_INFO: `/v3/projects/9dafcd0df8524413b4fe81a0931323b1/users/e60a5e751f22450c8e366c85b37af1f6/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.368103 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794634b1-e598-439f-b41e-a71dc93a86ed admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.375372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31af9b-5b98-4a73-9572-bdd528ff3a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.375873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31af9b-5b98-4a73-9572-bdd528ff3a33 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.376156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31af9b-5b98-4a73-9572-bdd528ff3a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.376435 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31af9b-5b98-4a73-9572-bdd528ff3a33 admin admin] PATH_INFO: `/v3/projects/29631c5dc5384b14a382e356cbfcda89/users/7b6fedc4f0cd428ebf8f1963ee81db37/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.385226 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f40f88-0b56-4cd8-b5a2-153b551fdc25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dafcd0df8524413b4fe81a0931323b1, user_id=e60a5e751f22450c8e366c85b37af1f6, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.388015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f40f88-0b56-4cd8-b5a2-153b551fdc25 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.394789 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31af9b-5b98-4a73-9572-bdd528ff3a33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29631c5dc5384b14a382e356cbfcda89, user_id=7b6fedc4f0cd428ebf8f1963ee81db37, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.396300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: WARNING py.warnings [None req-dc31af9b-5b98-4a73-9572-bdd528ff3a33 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:55:58.396300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: warnings.warn(msg) May 25 14:55:58.396300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: May 25 14:55:58.398568 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31af9b-5b98-4a73-9572-bdd528ff3a33 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.409401 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 795/1598] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:55:58 2022] PUT /identity/v3/projects/9dafcd0df8524413b4fe81a0931323b1/users/e60a5e751f22450c8e366c85b37af1f6/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:58.410683 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f40f88-0b56-4cd8-b5a2-153b551fdc25 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.416870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 804/1599] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:55:58 2022] PUT /identity/v3/projects/29631c5dc5384b14a382e356cbfcda89/users/7b6fedc4f0cd428ebf8f1963ee81db37/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:58.417721 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31af9b-5b98-4a73-9572-bdd528ff3a33 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.497561 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f40f88-0b56-4cd8-b5a2-153b551fdc25 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.497784 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31af9b-5b98-4a73-9572-bdd528ff3a33 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.503264 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd5ab1-2c1c-4794-b663-eb83ecaf662b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.503451 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33e441-ed8d-4296-8ea6-b3f127c98198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.503890 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd5ab1-2c1c-4794-b663-eb83ecaf662b admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.504001 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33e441-ed8d-4296-8ea6-b3f127c98198 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.504158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33e441-ed8d-4296-8ea6-b3f127c98198 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.504271 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd5ab1-2c1c-4794-b663-eb83ecaf662b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.504431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33e441-ed8d-4296-8ea6-b3f127c98198 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.504526 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd5ab1-2c1c-4794-b663-eb83ecaf662b admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.505141 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33e441-ed8d-4296-8ea6-b3f127c98198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.505273 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbd5ab1-2c1c-4794-b663-eb83ecaf662b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.506796 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33e441-ed8d-4296-8ea6-b3f127c98198 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.506913 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbd5ab1-2c1c-4794-b663-eb83ecaf662b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.520868 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 796/1600] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:58.520868 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33e441-ed8d-4296-8ea6-b3f127c98198 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.521612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 805/1601] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:58.521612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd5ab1-2c1c-4794-b663-eb83ecaf662b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.533379 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd5ab1-2c1c-4794-b663-eb83ecaf662b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.534728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33e441-ed8d-4296-8ea6-b3f127c98198 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.539487 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652d5ac-4a3e-421f-9b42-9d5e8b51253c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.539934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4e65e-066e-49df-9634-93d292142c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.540285 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652d5ac-4a3e-421f-9b42-9d5e8b51253c admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.540869 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652d5ac-4a3e-421f-9b42-9d5e8b51253c admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.540869 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652d5ac-4a3e-421f-9b42-9d5e8b51253c admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.541297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4e65e-066e-49df-9634-93d292142c06 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.541552 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652d5ac-4a3e-421f-9b42-9d5e8b51253c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.541662 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4e65e-066e-49df-9634-93d292142c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.541990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4e65e-066e-49df-9634-93d292142c06 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.542830 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4e65e-066e-49df-9634-93d292142c06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.543339 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652d5ac-4a3e-421f-9b42-9d5e8b51253c admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.546469 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4e65e-066e-49df-9634-93d292142c06 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.557296 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 797/1602] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:58.557559 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 806/1603] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:58.558599 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652d5ac-4a3e-421f-9b42-9d5e8b51253c admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.559167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d577345-ef29-465c-9e28-a3c02d6db032 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.559457 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d577345-ef29-465c-9e28-a3c02d6db032 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.559784 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d577345-ef29-465c-9e28-a3c02d6db032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.570384 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652d5ac-4a3e-421f-9b42-9d5e8b51253c admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.575489 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa0478-8b2d-4811-bb13-659fc7a0e1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.576101 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa0478-8b2d-4811-bb13-659fc7a0e1aa admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.576431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa0478-8b2d-4811-bb13-659fc7a0e1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.577282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa0478-8b2d-4811-bb13-659fc7a0e1aa admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.577957 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa0478-8b2d-4811-bb13-659fc7a0e1aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.582911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa0478-8b2d-4811-bb13-659fc7a0e1aa admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.587332 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-7d577345-ef29-465c-9e28-a3c02d6db032 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:58.589265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7d577345-ef29-465c-9e28-a3c02d6db032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:58.598995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 798/1604] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:58.600824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4c5ef-02f1-4331-8404-a23748c04b44 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.601011 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4c5ef-02f1-4331-8404-a23748c04b44 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.601502 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4c5ef-02f1-4331-8404-a23748c04b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.619211 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 807/1605] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:58.620541 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d577345-ef29-465c-9e28-a3c02d6db032 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.636173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d577345-ef29-465c-9e28-a3c02d6db032 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.642374 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-0bf4c5ef-02f1-4331-8404-a23748c04b44 None None] MFA Rules not processed for user `7b6fedc4f0cd428ebf8f1963ee81db37`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:58.644247 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-0bf4c5ef-02f1-4331-8404-a23748c04b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:58.644903 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc84163-2a54-4042-93f6-0a53a0ce5d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.646688 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc84163-2a54-4042-93f6-0a53a0ce5d7d admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.647012 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc84163-2a54-4042-93f6-0a53a0ce5d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.647287 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc84163-2a54-4042-93f6-0a53a0ce5d7d admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.648043 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc84163-2a54-4042-93f6-0a53a0ce5d7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.651766 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc84163-2a54-4042-93f6-0a53a0ce5d7d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.668269 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 808/1606] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:58.669374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc84163-2a54-4042-93f6-0a53a0ce5d7d admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.683857 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc84163-2a54-4042-93f6-0a53a0ce5d7d admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.694436 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3b3ce-aedd-4f9c-90aa-7d021341e518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.695761 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3b3ce-aedd-4f9c-90aa-7d021341e518 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.695761 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3b3ce-aedd-4f9c-90aa-7d021341e518 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.695979 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3b3ce-aedd-4f9c-90aa-7d021341e518 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.696711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e3b3ce-aedd-4f9c-90aa-7d021341e518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.698406 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e3b3ce-aedd-4f9c-90aa-7d021341e518 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.700435 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:58.716324 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 809/1607] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:55:58.717129 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3b3ce-aedd-4f9c-90aa-7d021341e518 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.729466 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3b3ce-aedd-4f9c-90aa-7d021341e518 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.735087 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff8b1a-5511-4b42-8adc-d04a1be5166b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.736007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff8b1a-5511-4b42-8adc-d04a1be5166b admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.736289 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff8b1a-5511-4b42-8adc-d04a1be5166b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.736567 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff8b1a-5511-4b42-8adc-d04a1be5166b admin admin] PATH_INFO: `/v3/projects/d7a4c63fb6c6476e8f6da76937c9ccd6/users/296dba10cbd54540bd9684f57b8e2099/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.753458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 799/1608] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:55:58 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:58.754353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4c5ef-02f1-4331-8404-a23748c04b44 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.759734 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff8b1a-5511-4b42-8adc-d04a1be5166b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7a4c63fb6c6476e8f6da76937c9ccd6, user_id=296dba10cbd54540bd9684f57b8e2099, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.765400 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff8b1a-5511-4b42-8adc-d04a1be5166b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.769000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:58.776311 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4c5ef-02f1-4331-8404-a23748c04b44 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.790505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 810/1609] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:55:58 2022] PUT /identity/v3/projects/d7a4c63fb6c6476e8f6da76937c9ccd6/users/296dba10cbd54540bd9684f57b8e2099/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:58.793282 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff8b1a-5511-4b42-8adc-d04a1be5166b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.856914 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c208f4-d7c4-4d9e-9b49-1f8bd20dc35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.858588 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c208f4-d7c4-4d9e-9b49-1f8bd20dc35b admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.859119 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c208f4-d7c4-4d9e-9b49-1f8bd20dc35b admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.859539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c208f4-d7c4-4d9e-9b49-1f8bd20dc35b admin admin] PATH_INFO: `/v3/projects/5fbc0fd1768644eab3fcabc81eed84f6/users/3393d62d88f74e80901615e71d8cbffb/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.876500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c208f4-d7c4-4d9e-9b49-1f8bd20dc35b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fbc0fd1768644eab3fcabc81eed84f6, user_id=3393d62d88f74e80901615e71d8cbffb, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.877148 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff8b1a-5511-4b42-8adc-d04a1be5166b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:58.880686 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c208f4-d7c4-4d9e-9b49-1f8bd20dc35b admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.883135 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6ba5a-7df9-4b81-b7cf-a7acda65d9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:58.885752 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6ba5a-7df9-4b81-b7cf-a7acda65d9a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:58.886256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6ba5a-7df9-4b81-b7cf-a7acda65d9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:58.886940 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6ba5a-7df9-4b81-b7cf-a7acda65d9a4 admin admin] PATH_INFO: `/v3/projects/acd27057280d4bdda5b9a689a1e399ff/users/97011bf8f6d444468fecdde7acfa1020/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:58.909700 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 800/1610] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:55:58 2022] PUT /identity/v3/projects/5fbc0fd1768644eab3fcabc81eed84f6/users/3393d62d88f74e80901615e71d8cbffb/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:58.909700 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c208f4-d7c4-4d9e-9b49-1f8bd20dc35b admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.919388 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6ba5a-7df9-4b81-b7cf-a7acda65d9a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acd27057280d4bdda5b9a689a1e399ff, user_id=97011bf8f6d444468fecdde7acfa1020, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:58.928938 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6ba5a-7df9-4b81-b7cf-a7acda65d9a4 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:58.949740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 811/1611] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:55:58 2022] PUT /identity/v3/projects/acd27057280d4bdda5b9a689a1e399ff/users/97011bf8f6d444468fecdde7acfa1020/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:58.952586 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6ba5a-7df9-4b81-b7cf-a7acda65d9a4 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:58.982367 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c208f4-d7c4-4d9e-9b49-1f8bd20dc35b admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:59.034605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6ba5a-7df9-4b81-b7cf-a7acda65d9a4 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:59.040876 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9eba8-df0d-4ff6-80e6-82f854793e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:59.041865 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9eba8-df0d-4ff6-80e6-82f854793e39 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:59.042362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9eba8-df0d-4ff6-80e6-82f854793e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:59.042707 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9eba8-df0d-4ff6-80e6-82f854793e39 admin admin] PATH_INFO: `/v3/projects/0eb91a551f3a47d3aac0d8003eeac7e8/users/03ddd6d7c04a4f96a2165944fab4aabf/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:59.060927 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad9eba8-df0d-4ff6-80e6-82f854793e39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eb91a551f3a47d3aac0d8003eeac7e8, user_id=03ddd6d7c04a4f96a2165944fab4aabf, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:59.061599 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0879d64-c6b0-485e-a929-b4e5883044b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:59.062386 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0879d64-c6b0-485e-a929-b4e5883044b8 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:59.062745 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0879d64-c6b0-485e-a929-b4e5883044b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:59.063149 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0879d64-c6b0-485e-a929-b4e5883044b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:59.064362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad9eba8-df0d-4ff6-80e6-82f854793e39 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:59.074377 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0879d64-c6b0-485e-a929-b4e5883044b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:59.076067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0879d64-c6b0-485e-a929-b4e5883044b8 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:59.086193 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 812/1612] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:55:58 2022] PUT /identity/v3/projects/0eb91a551f3a47d3aac0d8003eeac7e8/users/03ddd6d7c04a4f96a2165944fab4aabf/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:59.088398 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9eba8-df0d-4ff6-80e6-82f854793e39 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:59.090354 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 801/1613] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:55:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:59.092922 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0879d64-c6b0-485e-a929-b4e5883044b8 service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:59.142741 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:59.154149 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:59.168590 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9eba8-df0d-4ff6-80e6-82f854793e39 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:59.169172 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0879d64-c6b0-485e-a929-b4e5883044b8 service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:59.173756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054d733-5d9f-44d4-a98b-ad209d2d0189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:59.174735 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054d733-5d9f-44d4-a98b-ad209d2d0189 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:59.175019 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054d733-5d9f-44d4-a98b-ad209d2d0189 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:59.175296 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054d733-5d9f-44d4-a98b-ad209d2d0189 admin admin] PATH_INFO: `/v3/projects/d3fab2e6427d486bb1744239b759c415/users/b3cd4cec00fe4f0a8dafba7dc16d685e/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:59.207633 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1054d733-5d9f-44d4-a98b-ad209d2d0189 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3fab2e6427d486bb1744239b759c415, user_id=b3cd4cec00fe4f0a8dafba7dc16d685e, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:59.210591 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1054d733-5d9f-44d4-a98b-ad209d2d0189 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:59.228591 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 813/1614] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:55:59 2022] PUT /identity/v3/projects/d3fab2e6427d486bb1744239b759c415/users/b3cd4cec00fe4f0a8dafba7dc16d685e/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:55:59.230294 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69df732-d1fb-45a9-afc0-9ff93fe65cb4 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:59.230642 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69df732-d1fb-45a9-afc0-9ff93fe65cb4 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:59.231173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69df732-d1fb-45a9-afc0-9ff93fe65cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:59.249786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-d69df732-d1fb-45a9-afc0-9ff93fe65cb4 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:59.251201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d69df732-d1fb-45a9-afc0-9ff93fe65cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:59.276328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b6e31-eefe-47cf-9afa-1aec21602df7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:59.277669 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b6e31-eefe-47cf-9afa-1aec21602df7 service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:59.278915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b6e31-eefe-47cf-9afa-1aec21602df7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:59.280582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b6e31-eefe-47cf-9afa-1aec21602df7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:59.330545 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 814/1615] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:59.332799 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd99e15-fa6e-45aa-88da-14868bf8ebfa None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:59.333368 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd99e15-fa6e-45aa-88da-14868bf8ebfa None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:59.334442 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd99e15-fa6e-45aa-88da-14868bf8ebfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:59.374702 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-8cd99e15-fa6e-45aa-88da-14868bf8ebfa None None] MFA Rules not processed for user `3393d62d88f74e80901615e71d8cbffb`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:59.376290 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8cd99e15-fa6e-45aa-88da-14868bf8ebfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:59.390777 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b6e31-eefe-47cf-9afa-1aec21602df7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:59.392884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b6e31-eefe-47cf-9afa-1aec21602df7 service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:59.416881 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 802/1616] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:55:59 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:59.446497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a705f1a-8e33-47f2-b8f2-f76624586747 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:59.446887 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a705f1a-8e33-47f2-b8f2-f76624586747 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:59.447160 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a705f1a-8e33-47f2-b8f2-f76624586747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:59.479684 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 815/1617] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:55:59 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:59.481227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1b7e4-801c-47ad-87cb-424393ba870e None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:59.481674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1b7e4-801c-47ad-87cb-424393ba870e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:59.482244 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1b7e4-801c-47ad-87cb-424393ba870e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:59.484285 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-3a705f1a-8e33-47f2-b8f2-f76624586747 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:59.488337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-3a705f1a-8e33-47f2-b8f2-f76624586747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:59.509318 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 803/1618] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:59.516042 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b8f1b7e4-801c-47ad-87cb-424393ba870e None None] MFA Rules not processed for user `03ddd6d7c04a4f96a2165944fab4aabf`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:59.516488 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8bd2c-a44b-4812-acc0-0fd2cfe3199d None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:59.517019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8bd2c-a44b-4812-acc0-0fd2cfe3199d None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:59.517257 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b8f1b7e4-801c-47ad-87cb-424393ba870e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:59.517772 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8bd2c-a44b-4812-acc0-0fd2cfe3199d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:59.558436 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-f7c8bd2c-a44b-4812-acc0-0fd2cfe3199d None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:55:59.561999 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f7c8bd2c-a44b-4812-acc0-0fd2cfe3199d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:59.574165 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:59.583167 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 804/1619] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:59.584759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8bd2c-a44b-4812-acc0-0fd2cfe3199d None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:59.601443 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f7c8bd2c-a44b-4812-acc0-0fd2cfe3199d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:59.624566 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 816/1620] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:55:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:55:59.625621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1b7e4-801c-47ad-87cb-424393ba870e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:59.638084 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1b7e4-801c-47ad-87cb-424393ba870e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:59.644422 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f3f39-b4e2-4cae-8a82-e5edf1e2f0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:59.645214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f3f39-b4e2-4cae-8a82-e5edf1e2f0ee admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:59.646042 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f3f39-b4e2-4cae-8a82-e5edf1e2f0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:59.646475 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f3f39-b4e2-4cae-8a82-e5edf1e2f0ee admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:59.647604 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f3f39-b4e2-4cae-8a82-e5edf1e2f0ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:59.650154 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f3f39-b4e2-4cae-8a82-e5edf1e2f0ee admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:59.687363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 817/1621] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:55:59 2022] POST /identity/v3/projects => generated 355 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:55:59.688569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f3f39-b4e2-4cae-8a82-e5edf1e2f0ee admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:59.690576 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-155f3f39-b4e2-4cae-8a82-e5edf1e2f0ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:55:59.713706 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8bd2c-a44b-4812-acc0-0fd2cfe3199d None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:59.798569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f3f39-b4e2-4cae-8a82-e5edf1e2f0ee admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:59.810247 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca23ad-a883-4137-8fd0-71b32f324e17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:59.811242 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca23ad-a883-4137-8fd0-71b32f324e17 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:59.813126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca23ad-a883-4137-8fd0-71b32f324e17 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:59.813681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca23ad-a883-4137-8fd0-71b32f324e17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:59.863888 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e5106-37e5-4657-ab15-0f1a3af6f6d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:59.864592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e5106-37e5-4657-ab15-0f1a3af6f6d9 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:59.864882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e5106-37e5-4657-ab15-0f1a3af6f6d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:59.865149 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e5106-37e5-4657-ab15-0f1a3af6f6d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:59.907664 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:59.944677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca23ad-a883-4137-8fd0-71b32f324e17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:59.959676 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca23ad-a883-4137-8fd0-71b32f324e17 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:59.979894 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 818/1622] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:55:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:59.981364 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca23ad-a883-4137-8fd0-71b32f324e17 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:00.007316 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca23ad-a883-4137-8fd0-71b32f324e17 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:00.011276 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba98c57-0bd5-4506-9ff0-61642b07f3c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:00.011937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba98c57-0bd5-4506-9ff0-61642b07f3c1 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.012223 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba98c57-0bd5-4506-9ff0-61642b07f3c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.012498 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba98c57-0bd5-4506-9ff0-61642b07f3c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.025870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba98c57-0bd5-4506-9ff0-61642b07f3c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:00.027754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba98c57-0bd5-4506-9ff0-61642b07f3c1 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:00.051506 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 819/1623] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:55:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:00.053454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547714f-9d43-49f3-b249-4e2b0c00ff61 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.055640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547714f-9d43-49f3-b249-4e2b0c00ff61 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.055968 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547714f-9d43-49f3-b249-4e2b0c00ff61 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.057797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 820/1624] 10.222.0.78 () {54 vars in 961 bytes} [Wed May 25 14:56:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:56:00.059658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c547714f-9d43-49f3-b249-4e2b0c00ff61 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:00.074838 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e5106-37e5-4657-ab15-0f1a3af6f6d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:00.076883 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e5106-37e5-4657-ab15-0f1a3af6f6d9 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:00.084697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c547714f-9d43-49f3-b249-4e2b0c00ff61 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:00.097997 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee96af0-d9b2-48b4-9099-ded8ebe91ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:00.098767 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee96af0-d9b2-48b4-9099-ded8ebe91ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.099053 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee96af0-d9b2-48b4-9099-ded8ebe91ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.099330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee96af0-d9b2-48b4-9099-ded8ebe91ec6 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.101590 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 805/1625] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:55:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:00.102327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee96af0-d9b2-48b4-9099-ded8ebe91ec6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:00.105027 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b673f6-2048-4cf4-9e54-e38dae364909 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.106435 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee96af0-d9b2-48b4-9099-ded8ebe91ec6 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:00.107518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b673f6-2048-4cf4-9e54-e38dae364909 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.107876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b673f6-2048-4cf4-9e54-e38dae364909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.150632 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-71b673f6-2048-4cf4-9e54-e38dae364909 None None] MFA Rules not processed for user `e34e1d7ffbac49b0bde45829af0c6925`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:00.152726 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-71b673f6-2048-4cf4-9e54-e38dae364909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:00.157107 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 821/1626] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:56:00 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:00.159845 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee96af0-d9b2-48b4-9099-ded8ebe91ec6 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:00.181605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee96af0-d9b2-48b4-9099-ded8ebe91ec6 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:00.194993 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f52d42-66b5-470b-bfe7-bae88df29895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:00.195698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f52d42-66b5-470b-bfe7-bae88df29895 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.195986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f52d42-66b5-470b-bfe7-bae88df29895 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.196266 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f52d42-66b5-470b-bfe7-bae88df29895 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.197024 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f52d42-66b5-470b-bfe7-bae88df29895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:00.199522 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f52d42-66b5-470b-bfe7-bae88df29895 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:00.229011 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 822/1627] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:56:00.232394 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f52d42-66b5-470b-bfe7-bae88df29895 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:00.249185 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f52d42-66b5-470b-bfe7-bae88df29895 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:00.257422 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94349929-f1ed-4ad5-8f08-f96762489d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:00.261227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94349929-f1ed-4ad5-8f08-f96762489d80 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.261753 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94349929-f1ed-4ad5-8f08-f96762489d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.262624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94349929-f1ed-4ad5-8f08-f96762489d80 admin admin] PATH_INFO: `/v3/projects/0283df17437544bbb5b3452480958c06/users/cdfeeeabebd94fd78ecc803679e61dbf/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.303821 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 806/1628] 10.222.0.78 () {58 vars in 1058 bytes} [Wed May 25 14:56:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:00.313379 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b673f6-2048-4cf4-9e54-e38dae364909 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:00.316319 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94349929-f1ed-4ad5-8f08-f96762489d80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0283df17437544bbb5b3452480958c06, user_id=cdfeeeabebd94fd78ecc803679e61dbf, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:00.319223 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94349929-f1ed-4ad5-8f08-f96762489d80 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:00.334769 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b673f6-2048-4cf4-9e54-e38dae364909 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:00.340481 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 823/1629] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:00 2022] PUT /identity/v3/projects/0283df17437544bbb5b3452480958c06/users/cdfeeeabebd94fd78ecc803679e61dbf/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:00.344644 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94349929-f1ed-4ad5-8f08-f96762489d80 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:00.351076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:00.489217 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94349929-f1ed-4ad5-8f08-f96762489d80 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:00.495190 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c321e-2710-4756-9a02-116c67e08117 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:00.495885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c321e-2710-4756-9a02-116c67e08117 service nova] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.496168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c321e-2710-4756-9a02-116c67e08117 service nova] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.496445 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c321e-2710-4756-9a02-116c67e08117 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.500311 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c640a25-05b6-484b-a4c1-ce85a1fdd3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:00.508743 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c640a25-05b6-484b-a4c1-ce85a1fdd3eb admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.510589 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c640a25-05b6-484b-a4c1-ce85a1fdd3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.510885 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c640a25-05b6-484b-a4c1-ce85a1fdd3eb admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.513456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c640a25-05b6-484b-a4c1-ce85a1fdd3eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:00.517529 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c640a25-05b6-484b-a4c1-ce85a1fdd3eb admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:00.544398 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 824/1630] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:56:00.555884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c640a25-05b6-484b-a4c1-ce85a1fdd3eb admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:00.563876 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c640a25-05b6-484b-a4c1-ce85a1fdd3eb admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:00.572711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730faeef-2c79-4959-9133-7cc11a798163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:00.575030 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730faeef-2c79-4959-9133-7cc11a798163 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.576036 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730faeef-2c79-4959-9133-7cc11a798163 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.576711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730faeef-2c79-4959-9133-7cc11a798163 admin admin] PATH_INFO: `/v3/projects/0283df17437544bbb5b3452480958c06/users/cdfeeeabebd94fd78ecc803679e61dbf/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.596126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730faeef-2c79-4959-9133-7cc11a798163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0283df17437544bbb5b3452480958c06, user_id=cdfeeeabebd94fd78ecc803679e61dbf, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:00.600203 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:00.604475 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730faeef-2c79-4959-9133-7cc11a798163 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:00.633853 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 825/1631] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:00 2022] PUT /identity/v3/projects/0283df17437544bbb5b3452480958c06/users/cdfeeeabebd94fd78ecc803679e61dbf/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:00.636128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5e28e-27ef-40ae-a5d0-8b3d10527f50 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:00.636707 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5e28e-27ef-40ae-a5d0-8b3d10527f50 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:00.637250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5e28e-27ef-40ae-a5d0-8b3d10527f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:00.680424 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699c321e-2710-4756-9a02-116c67e08117 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:00.683190 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699c321e-2710-4756-9a02-116c67e08117 service nova] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:00.688462 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-15a5e28e-27ef-40ae-a5d0-8b3d10527f50 None None] MFA Rules not processed for user `cdfeeeabebd94fd78ecc803679e61dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:00.693093 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-15a5e28e-27ef-40ae-a5d0-8b3d10527f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:00.712577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 807/1632] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:00.831655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 826/1633] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:00 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:00.846795 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c321e-2710-4756-9a02-116c67e08117 service nova] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:00.973060 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c321e-2710-4756-9a02-116c67e08117 service nova] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:01.052932 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb238d5-8786-47eb-b50f-9cffe9653ef0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:01.054432 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb238d5-8786-47eb-b50f-9cffe9653ef0 service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:01.055780 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb238d5-8786-47eb-b50f-9cffe9653ef0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:01.055780 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb238d5-8786-47eb-b50f-9cffe9653ef0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:01.066747 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb238d5-8786-47eb-b50f-9cffe9653ef0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:01.068340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb238d5-8786-47eb-b50f-9cffe9653ef0 service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:01.080490 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 808/1634] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:00 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:01.105221 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc809e1-196b-4ac3-912e-bc01337b405d None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:01.105630 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc809e1-196b-4ac3-912e-bc01337b405d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:01.105973 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc809e1-196b-4ac3-912e-bc01337b405d None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:01.107485 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 827/1635] 10.222.0.78 () {54 vars in 912 bytes} [Wed May 25 14:56:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:56:01.133362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e546129-6c74-4f7d-9705-6807b2e1afd9 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:01.133681 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e546129-6c74-4f7d-9705-6807b2e1afd9 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:01.133957 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e546129-6c74-4f7d-9705-6807b2e1afd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:01.170399 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-4e546129-6c74-4f7d-9705-6807b2e1afd9 None None] MFA Rules not processed for user `e34e1d7ffbac49b0bde45829af0c6925`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:01.171672 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-4e546129-6c74-4f7d-9705-6807b2e1afd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:01.247270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc809e1-196b-4ac3-912e-bc01337b405d None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:01.280971 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc809e1-196b-4ac3-912e-bc01337b405d None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:01.289777 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db40fb1-7cc0-4957-9872-9612a7fff668 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:01.291282 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db40fb1-7cc0-4957-9872-9612a7fff668 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:01.292614 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db40fb1-7cc0-4957-9872-9612a7fff668 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:01.293568 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db40fb1-7cc0-4957-9872-9612a7fff668 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:01.429843 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 809/1636] 10.222.0.78 () {58 vars in 1009 bytes} [Wed May 25 14:56:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:01.442911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e546129-6c74-4f7d-9705-6807b2e1afd9 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:01.456675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e546129-6c74-4f7d-9705-6807b2e1afd9 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:01.462856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a749f-0963-43cd-9ada-5372cb6028f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:01.464353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a749f-0963-43cd-9ada-5372cb6028f5 service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:01.464353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a749f-0963-43cd-9ada-5372cb6028f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:01.464353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a749f-0963-43cd-9ada-5372cb6028f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:01.497470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db40fb1-7cc0-4957-9872-9612a7fff668 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:01.505405 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db40fb1-7cc0-4957-9872-9612a7fff668 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:01.505816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a749f-0963-43cd-9ada-5372cb6028f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:01.508463 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a749f-0963-43cd-9ada-5372cb6028f5 service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:01.544159 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 828/1637] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:01 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:56:01.549485 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 810/1638] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:56:02.596438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7707eb-8412-4603-ba05-2c43353a4476 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:02.597044 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7707eb-8412-4603-ba05-2c43353a4476 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:02.598593 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7707eb-8412-4603-ba05-2c43353a4476 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:02.601561 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 829/1639] 10.222.0.78 () {54 vars in 961 bytes} [Wed May 25 14:56:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:56:02.613032 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901c6ed-410f-4ca2-b15e-5ee13ec78ebd None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:02.613581 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901c6ed-410f-4ca2-b15e-5ee13ec78ebd None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:02.613978 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901c6ed-410f-4ca2-b15e-5ee13ec78ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:02.643660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-e901c6ed-410f-4ca2-b15e-5ee13ec78ebd None None] MFA Rules not processed for user `e34e1d7ffbac49b0bde45829af0c6925`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:02.645142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-e901c6ed-410f-4ca2-b15e-5ee13ec78ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:02.662536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 811/1640] 10.222.0.78 () {58 vars in 1058 bytes} [Wed May 25 14:56:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:02.668371 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2056a36-64d2-45b9-a600-a14a50ea0bd1 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:02.668777 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2056a36-64d2-45b9-a600-a14a50ea0bd1 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:02.669247 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2056a36-64d2-45b9-a600-a14a50ea0bd1 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:02.672458 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 830/1641] 10.222.0.78 () {54 vars in 912 bytes} [Wed May 25 14:56:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:56:02.681143 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a104499-a510-43d2-b4bc-d99428a69a65 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:02.681821 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a104499-a510-43d2-b4bc-d99428a69a65 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:02.681821 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a104499-a510-43d2-b4bc-d99428a69a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:02.703822 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:02.741030 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-3a104499-a510-43d2-b4bc-d99428a69a65 None None] MFA Rules not processed for user `e34e1d7ffbac49b0bde45829af0c6925`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:02.742815 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-3a104499-a510-43d2-b4bc-d99428a69a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:02.772314 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 812/1642] 10.222.0.78 () {58 vars in 1009 bytes} [Wed May 25 14:56:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:02.787964 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2056a36-64d2-45b9-a600-a14a50ea0bd1 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:02.807394 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2056a36-64d2-45b9-a600-a14a50ea0bd1 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:02.818062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3782f-657f-4bb3-9f1e-a5b9a64cf97b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:02.818895 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3782f-657f-4bb3-9f1e-a5b9a64cf97b service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:02.821032 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3782f-657f-4bb3-9f1e-a5b9a64cf97b service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:02.821475 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3782f-657f-4bb3-9f1e-a5b9a64cf97b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:02.838871 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:02.843532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3782f-657f-4bb3-9f1e-a5b9a64cf97b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:02.847434 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3782f-657f-4bb3-9f1e-a5b9a64cf97b service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:02.881029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 831/1643] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:03.364014 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88dfbe-70d1-4053-9062-6b3265f1bb13 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:03.364014 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88dfbe-70d1-4053-9062-6b3265f1bb13 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:03.364014 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88dfbe-70d1-4053-9062-6b3265f1bb13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:03.438732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-8f88dfbe-70d1-4053-9062-6b3265f1bb13 None None] MFA Rules not processed for user `296dba10cbd54540bd9684f57b8e2099`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:03.439977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-8f88dfbe-70d1-4053-9062-6b3265f1bb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:03.598627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 813/1644] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:03 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:03.603145 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3782f-657f-4bb3-9f1e-a5b9a64cf97b service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:03.692170 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3782f-657f-4bb3-9f1e-a5b9a64cf97b service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:03.697236 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d073bb-6d88-4b6b-9af2-25e5b2aae398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:03.697914 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d073bb-6d88-4b6b-9af2-25e5b2aae398 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:03.698627 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d073bb-6d88-4b6b-9af2-25e5b2aae398 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:03.699194 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d073bb-6d88-4b6b-9af2-25e5b2aae398 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:03.700239 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d073bb-6d88-4b6b-9af2-25e5b2aae398 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:03.705478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d073bb-6d88-4b6b-9af2-25e5b2aae398 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:03.749966 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 832/1645] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:56:03 2022] POST /identity/v3/projects => generated 403 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:03.753849 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88dfbe-70d1-4053-9062-6b3265f1bb13 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:03.777432 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88dfbe-70d1-4053-9062-6b3265f1bb13 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:03.889706 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed8914-3a76-4846-b25a-e0ec432a2642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:03.890666 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed8914-3a76-4846-b25a-e0ec432a2642 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:03.891086 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed8914-3a76-4846-b25a-e0ec432a2642 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:03.891630 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed8914-3a76-4846-b25a-e0ec432a2642 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:03.892451 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed8914-3a76-4846-b25a-e0ec432a2642 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:03.896967 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed8914-3a76-4846-b25a-e0ec432a2642 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:03.932395 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 814/1646] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:56:03 2022] POST /identity/v3/users => generated 308 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:03.934188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d073bb-6d88-4b6b-9af2-25e5b2aae398 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:03.948770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d073bb-6d88-4b6b-9af2-25e5b2aae398 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:03.955157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c2725-f9f4-49c1-97ca-9ec6a808ad8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:03.956101 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c2725-f9f4-49c1-97ca-9ec6a808ad8b admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:03.956610 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c2725-f9f4-49c1-97ca-9ec6a808ad8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:03.957087 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c2725-f9f4-49c1-97ca-9ec6a808ad8b admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:03.958053 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c2725-f9f4-49c1-97ca-9ec6a808ad8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:03.960106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c2725-f9f4-49c1-97ca-9ec6a808ad8b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:03.974627 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 833/1647] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:56:03.979022 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed8914-3a76-4846-b25a-e0ec432a2642 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:03.991506 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed8914-3a76-4846-b25a-e0ec432a2642 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:03.997907 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6efa0-a86b-4b35-a1ae-7a362b7e951d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:03.999587 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6efa0-a86b-4b35-a1ae-7a362b7e951d admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:03.999871 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6efa0-a86b-4b35-a1ae-7a362b7e951d admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:04.000147 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6efa0-a86b-4b35-a1ae-7a362b7e951d admin admin] PATH_INFO: `/v3/projects/5bbe9f9991bf4f9b876015d1d146b066/users/d921627d3793431eaa7aefd6a3b8db22/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:04.019109 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf6efa0-a86b-4b35-a1ae-7a362b7e951d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bbe9f9991bf4f9b876015d1d146b066, user_id=d921627d3793431eaa7aefd6a3b8db22, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:04.022306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf6efa0-a86b-4b35-a1ae-7a362b7e951d admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:04.040007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 815/1648] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:03 2022] PUT /identity/v3/projects/5bbe9f9991bf4f9b876015d1d146b066/users/d921627d3793431eaa7aefd6a3b8db22/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:04.043712 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c2725-f9f4-49c1-97ca-9ec6a808ad8b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:04.056647 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c2725-f9f4-49c1-97ca-9ec6a808ad8b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:04.138463 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01741de5-ff15-4a42-a9c9-7e3b5f3488dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:04.139962 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01741de5-ff15-4a42-a9c9-7e3b5f3488dc admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:04.140381 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01741de5-ff15-4a42-a9c9-7e3b5f3488dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:04.140774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01741de5-ff15-4a42-a9c9-7e3b5f3488dc admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:04.142091 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01741de5-ff15-4a42-a9c9-7e3b5f3488dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:04.144945 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01741de5-ff15-4a42-a9c9-7e3b5f3488dc admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:04.159945 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 834/1649] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:04.164671 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6efa0-a86b-4b35-a1ae-7a362b7e951d admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:04.180415 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6efa0-a86b-4b35-a1ae-7a362b7e951d admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:04.189554 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232519d9-d2b7-4f6d-93a4-4b89bfdf84d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:04.192144 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232519d9-d2b7-4f6d-93a4-4b89bfdf84d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:04.192144 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232519d9-d2b7-4f6d-93a4-4b89bfdf84d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:04.192144 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232519d9-d2b7-4f6d-93a4-4b89bfdf84d1 admin admin] PATH_INFO: `/v3/projects/5bbe9f9991bf4f9b876015d1d146b066/users/d921627d3793431eaa7aefd6a3b8db22/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:04.196028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232519d9-d2b7-4f6d-93a4-4b89bfdf84d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bbe9f9991bf4f9b876015d1d146b066, user_id=d921627d3793431eaa7aefd6a3b8db22, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:04.200005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232519d9-d2b7-4f6d-93a4-4b89bfdf84d1 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:04.229199 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 816/1650] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:04 2022] PUT /identity/v3/projects/5bbe9f9991bf4f9b876015d1d146b066/users/d921627d3793431eaa7aefd6a3b8db22/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:56:04.648896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a48eb9-d947-4ff7-aec3-b6a99e183a99 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:04.649337 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a48eb9-d947-4ff7-aec3-b6a99e183a99 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:04.649558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a48eb9-d947-4ff7-aec3-b6a99e183a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:04.736427 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-20a48eb9-d947-4ff7-aec3-b6a99e183a99 None None] MFA Rules not processed for user `b3cd4cec00fe4f0a8dafba7dc16d685e`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:04.737691 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-20a48eb9-d947-4ff7-aec3-b6a99e183a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:04.890217 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:04.912741 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 835/1651] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:04.925955 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28eca6-50ef-4f89-8c78-fdcc49fd91a9 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:04.926435 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28eca6-50ef-4f89-8c78-fdcc49fd91a9 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:04.926435 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28eca6-50ef-4f89-8c78-fdcc49fd91a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:04.987263 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-7e28eca6-50ef-4f89-8c78-fdcc49fd91a9 None None] MFA Rules not processed for user `b3cd4cec00fe4f0a8dafba7dc16d685e`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:04.993201 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-7e28eca6-50ef-4f89-8c78-fdcc49fd91a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:05.015871 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 817/1652] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:05.030289 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a48eb9-d947-4ff7-aec3-b6a99e183a99 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:05.122239 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28eca6-50ef-4f89-8c78-fdcc49fd91a9 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:05.228327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a48eb9-d947-4ff7-aec3-b6a99e183a99 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:05.244353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48362d-8249-4899-9ec9-b36c6bc1c909 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:05.245042 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48362d-8249-4899-9ec9-b36c6bc1c909 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:05.245334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48362d-8249-4899-9ec9-b36c6bc1c909 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:05.245614 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48362d-8249-4899-9ec9-b36c6bc1c909 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:05.249820 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:05.271476 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d48362d-8249-4899-9ec9-b36c6bc1c909 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:05.275037 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d48362d-8249-4899-9ec9-b36c6bc1c909 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:05.309783 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 836/1653] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:05.318358 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28eca6-50ef-4f89-8c78-fdcc49fd91a9 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:05.324302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ead7f8-d363-475f-a630-6f512f42cf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:05.326693 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ead7f8-d363-475f-a630-6f512f42cf80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:05.326693 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ead7f8-d363-475f-a630-6f512f42cf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:05.327006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ead7f8-d363-475f-a630-6f512f42cf80 admin admin] PATH_INFO: `/v3/users/03ddd6d7c04a4f96a2165944fab4aabf` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:05.328631 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ead7f8-d363-475f-a630-6f512f42cf80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03ddd6d7c04a4f96a2165944fab4aabf)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:05.331620 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ead7f8-d363-475f-a630-6f512f42cf80 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:05.550357 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-55ead7f8-d363-475f-a630-6f512f42cf80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03ddd6d7c04a4f96a2165944fab4aabf'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:05.616152 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-55ead7f8-d363-475f-a630-6f512f42cf80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03ddd6d7c04a4f96a2165944fab4aabf'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:05.642736 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-55ead7f8-d363-475f-a630-6f512f42cf80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03ddd6d7c04a4f96a2165944fab4aabf'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:05.644670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-55ead7f8-d363-475f-a630-6f512f42cf80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03ddd6d7c04a4f96a2165944fab4aabf'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:05.646303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-55ead7f8-d363-475f-a630-6f512f42cf80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03ddd6d7c04a4f96a2165944fab4aabf'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:05.681225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 818/1654] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:56:05 2022] DELETE /identity/v3/users/03ddd6d7c04a4f96a2165944fab4aabf => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:56:05.686402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b96d8-1846-4379-b9a3-110c5eb6a213 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:05.686927 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b96d8-1846-4379-b9a3-110c5eb6a213 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:05.686927 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b96d8-1846-4379-b9a3-110c5eb6a213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:05.723433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-6c9b96d8-1846-4379-b9a3-110c5eb6a213 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:05.726312 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-6c9b96d8-1846-4379-b9a3-110c5eb6a213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:05.915058 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 837/1655] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:05.958316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ead7f8-d363-475f-a630-6f512f42cf80 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:05.960510 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-55ead7f8-d363-475f-a630-6f512f42cf80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:06.112815 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ead7f8-d363-475f-a630-6f512f42cf80 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:06.139871 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95b566-962f-4686-bedb-7582ae68dba1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:06.140502 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95b566-962f-4686-bedb-7582ae68dba1 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:06.141474 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95b566-962f-4686-bedb-7582ae68dba1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:06.141474 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95b566-962f-4686-bedb-7582ae68dba1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:06.173093 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b95b566-962f-4686-bedb-7582ae68dba1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:06.174724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b95b566-962f-4686-bedb-7582ae68dba1 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:06.214610 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 819/1656] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:06.674347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936d60c-dc84-4186-836a-361138084dad None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:06.675498 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936d60c-dc84-4186-836a-361138084dad None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:06.675847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936d60c-dc84-4186-836a-361138084dad None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:06.677612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 838/1657] 10.222.0.78 () {54 vars in 912 bytes} [Wed May 25 14:56:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:56:06.683149 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80089ccc-c133-4d50-99b0-de86f295124e None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:06.683660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80089ccc-c133-4d50-99b0-de86f295124e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:06.683998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80089ccc-c133-4d50-99b0-de86f295124e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:06.780072 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-80089ccc-c133-4d50-99b0-de86f295124e None None] MFA Rules not processed for user `70ad51a8ad68440da6618891e2e08cb8`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:06.781187 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-80089ccc-c133-4d50-99b0-de86f295124e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:06.797811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 820/1658] 10.222.0.78 () {58 vars in 1009 bytes} [Wed May 25 14:56:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:06.818052 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936d60c-dc84-4186-836a-361138084dad None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:06.821440 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4936d60c-dc84-4186-836a-361138084dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:06.846807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936d60c-dc84-4186-836a-361138084dad None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:06.855007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58075f0d-5dc8-43eb-9631-b481fd910441 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:06.856178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58075f0d-5dc8-43eb-9631-b481fd910441 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:06.856596 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58075f0d-5dc8-43eb-9631-b481fd910441 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:06.857002 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58075f0d-5dc8-43eb-9631-b481fd910441 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:06.876928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58075f0d-5dc8-43eb-9631-b481fd910441 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:06.881108 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58075f0d-5dc8-43eb-9631-b481fd910441 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:06.917792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 839/1659] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:07.863155 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7ae09-22a4-4875-8128-043f3f87fb29 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:07.863625 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7ae09-22a4-4875-8128-043f3f87fb29 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:07.863694 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7ae09-22a4-4875-8128-043f3f87fb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:07.962583 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-d4d7ae09-22a4-4875-8128-043f3f87fb29 None None] MFA Rules not processed for user `d921627d3793431eaa7aefd6a3b8db22`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:07.963819 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d4d7ae09-22a4-4875-8128-043f3f87fb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:08.030505 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:08.120803 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 821/1660] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:07 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:08.581340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181e5a7-9bb7-4161-9ff1-4e0780dfe22d None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:08.581913 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181e5a7-9bb7-4161-9ff1-4e0780dfe22d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:08.582360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181e5a7-9bb7-4161-9ff1-4e0780dfe22d None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:08.584464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 840/1661] 10.222.0.78 () {54 vars in 960 bytes} [Wed May 25 14:56:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:56:08.606612 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d00c44-4f11-4b02-955a-99998a619c5c None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:08.607178 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d00c44-4f11-4b02-955a-99998a619c5c None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:08.608399 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d00c44-4f11-4b02-955a-99998a619c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:08.718966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-62d00c44-4f11-4b02-955a-99998a619c5c None None] MFA Rules not processed for user `63789c0a113447ff90933d447cf5bac3`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:08.721467 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-62d00c44-4f11-4b02-955a-99998a619c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:08.880327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181e5a7-9bb7-4161-9ff1-4e0780dfe22d None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:08.884277 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8181e5a7-9bb7-4161-9ff1-4e0780dfe22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:09.003212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181e5a7-9bb7-4161-9ff1-4e0780dfe22d None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:09.005732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 822/1662] 10.222.0.78 () {58 vars in 1057 bytes} [Wed May 25 14:56:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:09.015143 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d00c44-4f11-4b02-955a-99998a619c5c None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:09.027763 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cbe54-182e-4115-945e-41e9fe3664e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:09.028948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cbe54-182e-4115-945e-41e9fe3664e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:09.030071 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cbe54-182e-4115-945e-41e9fe3664e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:09.030499 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cbe54-182e-4115-945e-41e9fe3664e7 admin admin] PATH_INFO: `/v3/projects/0eb91a551f3a47d3aac0d8003eeac7e8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:09.033907 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375cbe54-182e-4115-945e-41e9fe3664e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eb91a551f3a47d3aac0d8003eeac7e8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:09.036277 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375cbe54-182e-4115-945e-41e9fe3664e7 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:09.041815 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d00c44-4f11-4b02-955a-99998a619c5c None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:09.061996 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25641975-709d-4522-966e-b1404397b2d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63789c0a113447ff90933d447cf5bac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:09.062738 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25641975-709d-4522-966e-b1404397b2d1 service placement] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:09.063749 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25641975-709d-4522-966e-b1404397b2d1 service placement] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:09.064049 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25641975-709d-4522-966e-b1404397b2d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:09.068186 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-25641975-709d-4522-966e-b1404397b2d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:09.089854 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25641975-709d-4522-966e-b1404397b2d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:09.092679 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25641975-709d-4522-966e-b1404397b2d1 service placement] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:09.111671 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 823/1663] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:56:09.166693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-375cbe54-182e-4115-945e-41e9fe3664e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eb91a551f3a47d3aac0d8003eeac7e8', 'request_id': 'req-375cbe54-182e-4115-945e-41e9fe3664e7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:09.196125 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-375cbe54-182e-4115-945e-41e9fe3664e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eb91a551f3a47d3aac0d8003eeac7e8', 'request_id': 'req-375cbe54-182e-4115-945e-41e9fe3664e7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:09.210943 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-375cbe54-182e-4115-945e-41e9fe3664e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eb91a551f3a47d3aac0d8003eeac7e8', 'request_id': 'req-375cbe54-182e-4115-945e-41e9fe3664e7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:09.216239 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:09.247159 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-375cbe54-182e-4115-945e-41e9fe3664e7 admin admin] The token cache is being invalidate because project 0eb91a551f3a47d3aac0d8003eeac7e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:09.247455 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-375cbe54-182e-4115-945e-41e9fe3664e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:09.250605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-375cbe54-182e-4115-945e-41e9fe3664e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:09.266673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 841/1664] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:56:08 2022] DELETE /identity/v3/projects/0eb91a551f3a47d3aac0d8003eeac7e8 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:09.268049 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d2ffa-bbe6-4c3a-9f7d-537068cd1e8f None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:09.268353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d2ffa-bbe6-4c3a-9f7d-537068cd1e8f None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:09.268636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d2ffa-bbe6-4c3a-9f7d-537068cd1e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:09.330187 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-598d2ffa-bbe6-4c3a-9f7d-537068cd1e8f None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:09.332683 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-598d2ffa-bbe6-4c3a-9f7d-537068cd1e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:09.566603 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a21bb3-7d9c-4031-af51-21f108578149 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:09.567074 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a21bb3-7d9c-4031-af51-21f108578149 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:09.567494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a21bb3-7d9c-4031-af51-21f108578149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:09.570925 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 824/1665] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:56:09.621238 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-e7a21bb3-7d9c-4031-af51-21f108578149 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:09.624319 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e7a21bb3-7d9c-4031-af51-21f108578149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:09.643072 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d2ffa-bbe6-4c3a-9f7d-537068cd1e8f None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:09.653664 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 842/1666] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:09.659691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d2ffa-bbe6-4c3a-9f7d-537068cd1e8f None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:09.671837 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755fcb2-05e1-4636-b1a2-798050be0aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:09.673067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755fcb2-05e1-4636-b1a2-798050be0aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:09.673579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755fcb2-05e1-4636-b1a2-798050be0aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:09.673860 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755fcb2-05e1-4636-b1a2-798050be0aa6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:09.674709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e755fcb2-05e1-4636-b1a2-798050be0aa6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:56:09.675023 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e755fcb2-05e1-4636-b1a2-798050be0aa6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:09.677561 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e755fcb2-05e1-4636-b1a2-798050be0aa6 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:09.704002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 825/1667] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:56:09.713807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a21bb3-7d9c-4031-af51-21f108578149 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:09.729649 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a21bb3-7d9c-4031-af51-21f108578149 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:09.738883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e5a89-f1ed-4312-99eb-98ce4ba9941d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:09.739825 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e5a89-f1ed-4312-99eb-98ce4ba9941d admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:09.740259 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e5a89-f1ed-4312-99eb-98ce4ba9941d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:09.740755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e5a89-f1ed-4312-99eb-98ce4ba9941d admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:09.741737 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e5a89-f1ed-4312-99eb-98ce4ba9941d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:09.744113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e5a89-f1ed-4312-99eb-98ce4ba9941d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:09.790505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 843/1668] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:56:09 2022] POST /identity/v3/projects => generated 371 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:09.803997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755fcb2-05e1-4636-b1a2-798050be0aa6 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:09.851895 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755fcb2-05e1-4636-b1a2-798050be0aa6 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:10.012300 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:10.095596 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4944b21-01e6-4c51-a0c2-f25c598997a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:10.096322 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4944b21-01e6-4c51-a0c2-f25c598997a7 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:10.100983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4944b21-01e6-4c51-a0c2-f25c598997a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:10.101790 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4944b21-01e6-4c51-a0c2-f25c598997a7 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:10.102609 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4944b21-01e6-4c51-a0c2-f25c598997a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:10.113557 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4944b21-01e6-4c51-a0c2-f25c598997a7 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:10.183450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e5a89-f1ed-4312-99eb-98ce4ba9941d admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:10.193598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 826/1669] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:56:09 2022] POST /identity/v3/users => generated 286 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:10.200293 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e5a89-f1ed-4312-99eb-98ce4ba9941d admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:10.216125 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c20b8-b244-45be-bed1-c70e8b4163d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:10.217380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c20b8-b244-45be-bed1-c70e8b4163d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:10.218083 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c20b8-b244-45be-bed1-c70e8b4163d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:10.218661 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c20b8-b244-45be-bed1-c70e8b4163d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:10.220096 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c20b8-b244-45be-bed1-c70e8b4163d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:10.225567 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c20b8-b244-45be-bed1-c70e8b4163d7 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:10.265293 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4944b21-01e6-4c51-a0c2-f25c598997a7 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:10.280249 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 844/1670] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:56:10.297515 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4944b21-01e6-4c51-a0c2-f25c598997a7 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:10.322627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f971a-56ce-4416-82fc-770387b02eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:10.323754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f971a-56ce-4416-82fc-770387b02eaa admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:10.324064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f971a-56ce-4416-82fc-770387b02eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:10.327842 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f971a-56ce-4416-82fc-770387b02eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:10.328740 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f971a-56ce-4416-82fc-770387b02eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:10.335835 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f971a-56ce-4416-82fc-770387b02eaa admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:10.372898 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c20b8-b244-45be-bed1-c70e8b4163d7 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:10.392836 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 827/1671] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:10.407173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c20b8-b244-45be-bed1-c70e8b4163d7 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:10.424861 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e33e81-3c09-4e6a-86ce-4ed36a019ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:10.428107 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e33e81-3c09-4e6a-86ce-4ed36a019ffc admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:10.429629 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e33e81-3c09-4e6a-86ce-4ed36a019ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:10.439465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e33e81-3c09-4e6a-86ce-4ed36a019ffc admin admin] PATH_INFO: `/v3/projects/2d046f639c4f4aaa9ec3d05168d416f2/users/4db870f17c424976a5610f2a7f0fd0f5/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:10.483516 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e33e81-3c09-4e6a-86ce-4ed36a019ffc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d046f639c4f4aaa9ec3d05168d416f2, user_id=4db870f17c424976a5610f2a7f0fd0f5, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:10.493840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e33e81-3c09-4e6a-86ce-4ed36a019ffc admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:10.554667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 845/1672] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:10 2022] PUT /identity/v3/projects/2d046f639c4f4aaa9ec3d05168d416f2/users/4db870f17c424976a5610f2a7f0fd0f5/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:10.556759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90475053-454e-4052-8898-fa4c769ff087 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:10.557078 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90475053-454e-4052-8898-fa4c769ff087 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:10.557353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90475053-454e-4052-8898-fa4c769ff087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:10.626380 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-90475053-454e-4052-8898-fa4c769ff087 None None] MFA Rules not processed for user `4db870f17c424976a5610f2a7f0fd0f5`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:10.633254 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-90475053-454e-4052-8898-fa4c769ff087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:10.860522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 828/1673] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) May 25 14:56:10.867866 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e33e81-3c09-4e6a-86ce-4ed36a019ffc admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:11.047937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e33e81-3c09-4e6a-86ce-4ed36a019ffc admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:11.068416 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffea1ed-dbc5-4b2c-8450-4e1506c3f9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:11.069376 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffea1ed-dbc5-4b2c-8450-4e1506c3f9a3 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:11.069799 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffea1ed-dbc5-4b2c-8450-4e1506c3f9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:11.070315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffea1ed-dbc5-4b2c-8450-4e1506c3f9a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:11.071796 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffea1ed-dbc5-4b2c-8450-4e1506c3f9a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:11.077901 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffea1ed-dbc5-4b2c-8450-4e1506c3f9a3 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:11.135741 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 846/1674] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:56:10 2022] POST /identity/v3/projects => generated 371 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:11.138317 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90475053-454e-4052-8898-fa4c769ff087 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:11.151532 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90475053-454e-4052-8898-fa4c769ff087 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:11.224708 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa86f52-7018-4d11-8885-8ff078c64d03 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:11.225454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa86f52-7018-4d11-8885-8ff078c64d03 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:11.225454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa86f52-7018-4d11-8885-8ff078c64d03 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:11.227367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 847/1675] 10.222.0.78 () {54 vars in 921 bytes} [Wed May 25 14:56:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:56:11.258897 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a0266-46a2-4ebf-95ac-3b00aff199d0 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:11.259267 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a0266-46a2-4ebf-95ac-3b00aff199d0 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:11.259662 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a0266-46a2-4ebf-95ac-3b00aff199d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:11.271451 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae486db-9af9-4ade-9bf9-ba2740a93187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:11.272407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae486db-9af9-4ade-9bf9-ba2740a93187 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:11.272696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae486db-9af9-4ade-9bf9-ba2740a93187 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:11.272973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae486db-9af9-4ade-9bf9-ba2740a93187 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:11.275976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae486db-9af9-4ade-9bf9-ba2740a93187 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:11.287099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae486db-9af9-4ade-9bf9-ba2740a93187 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:11.324979 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-c91a0266-46a2-4ebf-95ac-3b00aff199d0 None None] MFA Rules not processed for user `63789c0a113447ff90933d447cf5bac3`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:11.327309 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c91a0266-46a2-4ebf-95ac-3b00aff199d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:11.346890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 829/1676] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:56:11 2022] POST /identity/v3/users => generated 292 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:11.348096 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae486db-9af9-4ade-9bf9-ba2740a93187 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:11.377437 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae486db-9af9-4ade-9bf9-ba2740a93187 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:11.383017 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f7074-1d24-40ae-85c0-fd36b8dee7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:11.383596 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f7074-1d24-40ae-85c0-fd36b8dee7fb admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:11.383888 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f7074-1d24-40ae-85c0-fd36b8dee7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:11.384275 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f7074-1d24-40ae-85c0-fd36b8dee7fb admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:11.385143 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f7074-1d24-40ae-85c0-fd36b8dee7fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:11.386751 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f7074-1d24-40ae-85c0-fd36b8dee7fb admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:11.422344 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 830/1677] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:11.429765 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f7074-1d24-40ae-85c0-fd36b8dee7fb admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:11.466297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f7074-1d24-40ae-85c0-fd36b8dee7fb admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:11.479905 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791f7d3-84f6-45c0-a3ea-3e43f6c79cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:11.482718 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791f7d3-84f6-45c0-a3ea-3e43f6c79cca admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:11.484466 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791f7d3-84f6-45c0-a3ea-3e43f6c79cca admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:11.487055 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791f7d3-84f6-45c0-a3ea-3e43f6c79cca admin admin] PATH_INFO: `/v3/projects/bde86e084d2a4e0caf9ac518a477ad09/users/fe05ae55efb646b88211609a47870eed/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:11.548869 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791f7d3-84f6-45c0-a3ea-3e43f6c79cca admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde86e084d2a4e0caf9ac518a477ad09, user_id=fe05ae55efb646b88211609a47870eed, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:11.551785 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791f7d3-84f6-45c0-a3ea-3e43f6c79cca admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:11.559761 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:11.590605 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 831/1678] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:11 2022] PUT /identity/v3/projects/bde86e084d2a4e0caf9ac518a477ad09/users/fe05ae55efb646b88211609a47870eed/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:11.594784 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791f7d3-84f6-45c0-a3ea-3e43f6c79cca admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:11.609004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 848/1679] 10.222.0.78 () {58 vars in 1018 bytes} [Wed May 25 14:56:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:11.768978 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791f7d3-84f6-45c0-a3ea-3e43f6c79cca admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:11.794134 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c993f7e-b06f-457f-9e85-575f7bcc60fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:11.794938 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c993f7e-b06f-457f-9e85-575f7bcc60fd admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:11.795385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c993f7e-b06f-457f-9e85-575f7bcc60fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:11.795672 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c993f7e-b06f-457f-9e85-575f7bcc60fd admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:11.799544 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c993f7e-b06f-457f-9e85-575f7bcc60fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:11.802329 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c993f7e-b06f-457f-9e85-575f7bcc60fd admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:11.822346 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 832/1680] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:11.827835 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a0266-46a2-4ebf-95ac-3b00aff199d0 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:11.828477 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c993f7e-b06f-457f-9e85-575f7bcc60fd admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:11.829717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c91a0266-46a2-4ebf-95ac-3b00aff199d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:11.843015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c993f7e-b06f-457f-9e85-575f7bcc60fd admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:11.857559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1fb0c9-1906-4bc0-bc92-7a602869bf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:11.858048 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a0266-46a2-4ebf-95ac-3b00aff199d0 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:11.859926 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1fb0c9-1906-4bc0-bc92-7a602869bf9c admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:11.860308 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1fb0c9-1906-4bc0-bc92-7a602869bf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:11.860621 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1fb0c9-1906-4bc0-bc92-7a602869bf9c admin admin] PATH_INFO: `/v3/projects/bde86e084d2a4e0caf9ac518a477ad09/users/fe05ae55efb646b88211609a47870eed/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:11.870674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:11.873677 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1fb0c9-1906-4bc0-bc92-7a602869bf9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde86e084d2a4e0caf9ac518a477ad09, user_id=fe05ae55efb646b88211609a47870eed, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:11.882746 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1fb0c9-1906-4bc0-bc92-7a602869bf9c admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:11.932003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 833/1681] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:11 2022] PUT /identity/v3/projects/bde86e084d2a4e0caf9ac518a477ad09/users/fe05ae55efb646b88211609a47870eed/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:11.942065 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c3040-c811-45bd-bb30-b7ec2c96ecce None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:11.942327 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c3040-c811-45bd-bb30-b7ec2c96ecce None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:11.942621 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c3040-c811-45bd-bb30-b7ec2c96ecce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:12.075763 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-a44c3040-c811-45bd-bb30-b7ec2c96ecce None None] MFA Rules not processed for user `fe05ae55efb646b88211609a47870eed`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:12.077015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-a44c3040-c811-45bd-bb30-b7ec2c96ecce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:12.084725 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729b9b8-1f22-4080-b755-dfbf0973673c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63789c0a113447ff90933d447cf5bac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:12.085407 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729b9b8-1f22-4080-b755-dfbf0973673c service placement] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:12.085698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729b9b8-1f22-4080-b755-dfbf0973673c service placement] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:12.085980 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729b9b8-1f22-4080-b755-dfbf0973673c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:12.116495 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729b9b8-1f22-4080-b755-dfbf0973673c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:12.118737 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729b9b8-1f22-4080-b755-dfbf0973673c service placement] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:12.166934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 849/1682] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:12.182200 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b98d8-ae22-4a08-bcf2-6a7ce004f016 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:12.199506 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b98d8-ae22-4a08-bcf2-6a7ce004f016 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:12.201325 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b98d8-ae22-4a08-bcf2-6a7ce004f016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:12.329234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b09b98d8-ae22-4a08-bcf2-6a7ce004f016 None None] MFA Rules not processed for user `7fedb9b92e9a4e3cb671f69751e426a1`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:12.333413 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b09b98d8-ae22-4a08-bcf2-6a7ce004f016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:12.501435 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 834/1683] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:11 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 560 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:56:12.512969 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c3040-c811-45bd-bb30-b7ec2c96ecce None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:12.516870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-a44c3040-c811-45bd-bb30-b7ec2c96ecce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:12.630881 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 850/1684] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:12.649652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b98d8-ae22-4a08-bcf2-6a7ce004f016 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:12.651246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b09b98d8-ae22-4a08-bcf2-6a7ce004f016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:12.707831 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c3040-c811-45bd-bb30-b7ec2c96ecce None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:12.725929 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7b137-ef05-4eea-a960-6763c5ceb4b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:12.727244 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7b137-ef05-4eea-a960-6763c5ceb4b2 service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:12.727531 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7b137-ef05-4eea-a960-6763c5ceb4b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:12.727807 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7b137-ef05-4eea-a960-6763c5ceb4b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:12.758327 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7b137-ef05-4eea-a960-6763c5ceb4b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:12.760982 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7b137-ef05-4eea-a960-6763c5ceb4b2 service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:12.826896 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 835/1685] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:12 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:56:12.874917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b98d8-ae22-4a08-bcf2-6a7ce004f016 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:12.882083 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7b137-ef05-4eea-a960-6763c5ceb4b2 service cinder] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:12.883648 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-0dc7b137-ef05-4eea-a960-6763c5ceb4b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:12.916304 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d25fc-0060-43ae-9a61-d4f3ac3b9adf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:12.917167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d25fc-0060-43ae-9a61-d4f3ac3b9adf service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:12.917481 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d25fc-0060-43ae-9a61-d4f3ac3b9adf service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:12.917782 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d25fc-0060-43ae-9a61-d4f3ac3b9adf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:12.930126 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:12.946672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d25fc-0060-43ae-9a61-d4f3ac3b9adf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:12.948333 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d25fc-0060-43ae-9a61-d4f3ac3b9adf service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:13.005245 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 851/1686] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:13.161291 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7b137-ef05-4eea-a960-6763c5ceb4b2 service cinder] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:13.194717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a23b4e-e30c-4fbc-b5e2-4309b591a315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:13.195622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a23b4e-e30c-4fbc-b5e2-4309b591a315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:13.195919 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a23b4e-e30c-4fbc-b5e2-4309b591a315 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:13.196224 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a23b4e-e30c-4fbc-b5e2-4309b591a315 admin admin] PATH_INFO: `/v3/users/7b6fedc4f0cd428ebf8f1963ee81db37` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:13.211465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a23b4e-e30c-4fbc-b5e2-4309b591a315 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b6fedc4f0cd428ebf8f1963ee81db37)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:13.213677 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a23b4e-e30c-4fbc-b5e2-4309b591a315 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:13.341370 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-41a23b4e-e30c-4fbc-b5e2-4309b591a315 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b6fedc4f0cd428ebf8f1963ee81db37'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:13.386778 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-41a23b4e-e30c-4fbc-b5e2-4309b591a315 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b6fedc4f0cd428ebf8f1963ee81db37'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:13.402996 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-41a23b4e-e30c-4fbc-b5e2-4309b591a315 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b6fedc4f0cd428ebf8f1963ee81db37'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:13.404182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-41a23b4e-e30c-4fbc-b5e2-4309b591a315 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b6fedc4f0cd428ebf8f1963ee81db37'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:13.405331 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-41a23b4e-e30c-4fbc-b5e2-4309b591a315 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b6fedc4f0cd428ebf8f1963ee81db37'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:13.436831 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334143fe-e854-42e3-bb41-4c1dfc3fbf6c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:13.436831 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334143fe-e854-42e3-bb41-4c1dfc3fbf6c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:13.437007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334143fe-e854-42e3-bb41-4c1dfc3fbf6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:13.459520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 836/1687] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:56:12 2022] DELETE /identity/v3/users/7b6fedc4f0cd428ebf8f1963ee81db37 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:13.459520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a18d0f-f01d-4090-8e14-653be0e67854 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:13.459817 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a18d0f-f01d-4090-8e14-653be0e67854 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:13.462186 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a18d0f-f01d-4090-8e14-653be0e67854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:13.513904 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-334143fe-e854-42e3-bb41-4c1dfc3fbf6c None None] MFA Rules not processed for user `b2935579c8554bed8b1e211c47548ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:13.513904 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-334143fe-e854-42e3-bb41-4c1dfc3fbf6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:13.528545 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-d7a18d0f-f01d-4090-8e14-653be0e67854 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:13.531617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d7a18d0f-f01d-4090-8e14-653be0e67854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:13.550936 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:13.676524 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 852/1688] 10.222.0.78 () {58 vars in 1009 bytes} [Wed May 25 14:56:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:13.682604 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334143fe-e854-42e3-bb41-4c1dfc3fbf6c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:13.687286 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-334143fe-e854-42e3-bb41-4c1dfc3fbf6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:13.692213 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 837/1689] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:13.739543 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334143fe-e854-42e3-bb41-4c1dfc3fbf6c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:13.752607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a909809-fe6a-461a-9222-105df53b5737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:13.753508 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a909809-fe6a-461a-9222-105df53b5737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:13.753791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a909809-fe6a-461a-9222-105df53b5737 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:13.754134 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a909809-fe6a-461a-9222-105df53b5737 admin admin] PATH_INFO: `/v3/users/4db870f17c424976a5610f2a7f0fd0f5` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:13.755539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a909809-fe6a-461a-9222-105df53b5737 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4db870f17c424976a5610f2a7f0fd0f5)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:13.756084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a18d0f-f01d-4090-8e14-653be0e67854 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:13.756591 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: WARNING py.warnings [None req-2a909809-fe6a-461a-9222-105df53b5737 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:56:13.756591 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: warnings.warn(msg) May 25 14:56:13.756591 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: May 25 14:56:13.757899 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a909809-fe6a-461a-9222-105df53b5737 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:13.758320 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d7a18d0f-f01d-4090-8e14-653be0e67854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:13.825084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:13.883626 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a18d0f-f01d-4090-8e14-653be0e67854 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:13.889529 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2a909809-fe6a-461a-9222-105df53b5737 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4db870f17c424976a5610f2a7f0fd0f5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:13.897160 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee55f7-0d49-46aa-b7d0-1bf21f2405dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:13.897160 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee55f7-0d49-46aa-b7d0-1bf21f2405dd service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:13.897160 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee55f7-0d49-46aa-b7d0-1bf21f2405dd service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:13.897160 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee55f7-0d49-46aa-b7d0-1bf21f2405dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:13.927039 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee55f7-0d49-46aa-b7d0-1bf21f2405dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:13.930317 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee55f7-0d49-46aa-b7d0-1bf21f2405dd service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:13.949838 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2a909809-fe6a-461a-9222-105df53b5737 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4db870f17c424976a5610f2a7f0fd0f5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:13.955370 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 838/1690] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:13.956622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee55f7-0d49-46aa-b7d0-1bf21f2405dd service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:13.959962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-75ee55f7-0d49-46aa-b7d0-1bf21f2405dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:13.961834 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2a909809-fe6a-461a-9222-105df53b5737 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4db870f17c424976a5610f2a7f0fd0f5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:13.963199 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2a909809-fe6a-461a-9222-105df53b5737 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4db870f17c424976a5610f2a7f0fd0f5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:13.965360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2a909809-fe6a-461a-9222-105df53b5737 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4db870f17c424976a5610f2a7f0fd0f5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:14.027071 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 853/1691] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:56:13 2022] DELETE /identity/v3/users/4db870f17c424976a5610f2a7f0fd0f5 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:14.034446 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31506849-605a-462c-8989-e3a086092c8f None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:14.037900 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31506849-605a-462c-8989-e3a086092c8f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:14.038337 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31506849-605a-462c-8989-e3a086092c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:14.142628 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-31506849-605a-462c-8989-e3a086092c8f None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:14.146540 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-31506849-605a-462c-8989-e3a086092c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:14.260243 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 854/1692] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:14.265298 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee55f7-0d49-46aa-b7d0-1bf21f2405dd service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:14.351334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31506849-605a-462c-8989-e3a086092c8f None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:14.355186 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-31506849-605a-462c-8989-e3a086092c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:14.430945 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b5705-f300-4076-9ba2-4d7cf25cc0b9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe4a2d106d84fc9ac65c046ba675353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:14.433507 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b5705-f300-4076-9ba2-4d7cf25cc0b9 service swift] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:14.435430 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b5705-f300-4076-9ba2-4d7cf25cc0b9 service swift] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:14.439577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b5705-f300-4076-9ba2-4d7cf25cc0b9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:14.444940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-168b5705-f300-4076-9ba2-4d7cf25cc0b9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:14.556314 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:14.649786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31506849-605a-462c-8989-e3a086092c8f None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:14.675080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094e20e-c69e-41a4-9a90-cff940a467d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:14.675764 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094e20e-c69e-41a4-9a90-cff940a467d6 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:14.676054 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094e20e-c69e-41a4-9a90-cff940a467d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:14.676339 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094e20e-c69e-41a4-9a90-cff940a467d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:14.705992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094e20e-c69e-41a4-9a90-cff940a467d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:14.707617 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094e20e-c69e-41a4-9a90-cff940a467d6 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:14.716856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b5705-f300-4076-9ba2-4d7cf25cc0b9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:14.724968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b5705-f300-4076-9ba2-4d7cf25cc0b9 service swift] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:14.738162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 855/1693] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:56:14.739802 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90200f91-7f11-4006-8ee5-98ef69c0c090 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:14.740247 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90200f91-7f11-4006-8ee5-98ef69c0c090 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:14.740672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90200f91-7f11-4006-8ee5-98ef69c0c090 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:14.746534 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 856/1694] 10.222.0.78 () {54 vars in 921 bytes} [Wed May 25 14:56:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:56:14.785841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 839/1695] 10.222.0.78 () {60 vars in 1384 bytes} [Wed May 25 14:56:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 828 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 14:56:14.990463 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8ee6e-1198-4545-a39e-e2c5cc091554 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:14.990992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8ee6e-1198-4545-a39e-e2c5cc091554 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:14.991158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8ee6e-1198-4545-a39e-e2c5cc091554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:15.051873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-8fe8ee6e-1198-4545-a39e-e2c5cc091554 None None] MFA Rules not processed for user `63789c0a113447ff90933d447cf5bac3`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:15.057746 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8fe8ee6e-1198-4545-a39e-e2c5cc091554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:15.345134 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 857/1696] 10.222.0.78 () {58 vars in 1018 bytes} [Wed May 25 14:56:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:15.363532 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b5705-f300-4076-9ba2-4d7cf25cc0b9 service swift] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:15.365327 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-168b5705-f300-4076-9ba2-4d7cf25cc0b9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:15.391795 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b5705-f300-4076-9ba2-4d7cf25cc0b9 service swift] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:15.405742 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463c43e-2330-4f05-a4dd-13731f049fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:15.414081 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463c43e-2330-4f05-a4dd-13731f049fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:15.419511 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463c43e-2330-4f05-a4dd-13731f049fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:15.420177 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463c43e-2330-4f05-a4dd-13731f049fa1 admin admin] PATH_INFO: `/v3/users/296dba10cbd54540bd9684f57b8e2099` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:15.422001 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463c43e-2330-4f05-a4dd-13731f049fa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=296dba10cbd54540bd9684f57b8e2099)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:15.430833 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463c43e-2330-4f05-a4dd-13731f049fa1 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:15.446371 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:15.494839 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe8ee6e-1198-4545-a39e-e2c5cc091554 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:15.502639 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8fe8ee6e-1198-4545-a39e-e2c5cc091554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:15.549187 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe8ee6e-1198-4545-a39e-e2c5cc091554 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:15.567859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c19bbc-d23c-4e90-8609-30ec61273c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:15.568827 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c19bbc-d23c-4e90-8609-30ec61273c17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:15.569140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c19bbc-d23c-4e90-8609-30ec61273c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:15.569444 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c19bbc-d23c-4e90-8609-30ec61273c17 admin admin] PATH_INFO: `/v3/projects/29631c5dc5384b14a382e356cbfcda89` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:15.577443 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c19bbc-d23c-4e90-8609-30ec61273c17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29631c5dc5384b14a382e356cbfcda89)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:15.584449 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c19bbc-d23c-4e90-8609-30ec61273c17 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:15.595537 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6463c43e-2330-4f05-a4dd-13731f049fa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '296dba10cbd54540bd9684f57b8e2099'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:15.640168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6463c43e-2330-4f05-a4dd-13731f049fa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '296dba10cbd54540bd9684f57b8e2099'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:15.669198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6463c43e-2330-4f05-a4dd-13731f049fa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '296dba10cbd54540bd9684f57b8e2099'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:15.689785 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6463c43e-2330-4f05-a4dd-13731f049fa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '296dba10cbd54540bd9684f57b8e2099'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:15.693171 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6463c43e-2330-4f05-a4dd-13731f049fa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '296dba10cbd54540bd9684f57b8e2099'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:15.849346 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 840/1697] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:56:15 2022] DELETE /identity/v3/users/296dba10cbd54540bd9684f57b8e2099 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:15.861349 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463c43e-2330-4f05-a4dd-13731f049fa1 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:15.870857 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-6463c43e-2330-4f05-a4dd-13731f049fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:15.926000 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-10c19bbc-d23c-4e90-8609-30ec61273c17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29631c5dc5384b14a382e356cbfcda89', 'request_id': 'req-10c19bbc-d23c-4e90-8609-30ec61273c17'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:15.968626 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-10c19bbc-d23c-4e90-8609-30ec61273c17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29631c5dc5384b14a382e356cbfcda89', 'request_id': 'req-10c19bbc-d23c-4e90-8609-30ec61273c17'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:16.040850 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-10c19bbc-d23c-4e90-8609-30ec61273c17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29631c5dc5384b14a382e356cbfcda89', 'request_id': 'req-10c19bbc-d23c-4e90-8609-30ec61273c17'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:16.095679 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-10c19bbc-d23c-4e90-8609-30ec61273c17 admin admin] The token cache is being invalidate because project 29631c5dc5384b14a382e356cbfcda89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:16.096137 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-10c19bbc-d23c-4e90-8609-30ec61273c17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:16.100133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-10c19bbc-d23c-4e90-8609-30ec61273c17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:16.103078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 858/1698] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:56:15 2022] DELETE /identity/v3/projects/29631c5dc5384b14a382e356cbfcda89 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:16.106380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317fcbef-76f6-4e90-8593-3ede776bb17e None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:16.106538 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317fcbef-76f6-4e90-8593-3ede776bb17e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:16.107432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317fcbef-76f6-4e90-8593-3ede776bb17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:16.153313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-317fcbef-76f6-4e90-8593-3ede776bb17e None None] MFA Rules not processed for user `97011bf8f6d444468fecdde7acfa1020`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:16.156043 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-317fcbef-76f6-4e90-8593-3ede776bb17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:16.265767 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463c43e-2330-4f05-a4dd-13731f049fa1 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:16.275885 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:16.281226 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f308f1-6579-4937-b578-cf4f2a55b94f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63789c0a113447ff90933d447cf5bac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:16.282164 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f308f1-6579-4937-b578-cf4f2a55b94f service placement] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:16.282290 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f308f1-6579-4937-b578-cf4f2a55b94f service placement] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:16.282513 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f308f1-6579-4937-b578-cf4f2a55b94f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:16.286259 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d5f308f1-6579-4937-b578-cf4f2a55b94f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:16.347235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f308f1-6579-4937-b578-cf4f2a55b94f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:16.352363 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f308f1-6579-4937-b578-cf4f2a55b94f service placement] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:16.386714 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 859/1699] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:16.397795 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dd658-de31-4267-a50a-3a7dbc4a802b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:16.400645 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dd658-de31-4267-a50a-3a7dbc4a802b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:16.401277 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dd658-de31-4267-a50a-3a7dbc4a802b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:16.415260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 841/1700] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:16.420859 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1736943-f91c-4e0a-a582-35c8633bee5a None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:16.422860 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1736943-f91c-4e0a-a582-35c8633bee5a None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:16.424812 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1736943-f91c-4e0a-a582-35c8633bee5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:16.488962 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-1b1dd658-de31-4267-a50a-3a7dbc4a802b None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:16.500218 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1b1dd658-de31-4267-a50a-3a7dbc4a802b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:16.534083 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-a1736943-f91c-4e0a-a582-35c8633bee5a None None] MFA Rules not processed for user `97011bf8f6d444468fecdde7acfa1020`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:16.540005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-a1736943-f91c-4e0a-a582-35c8633bee5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:16.613558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 842/1701] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:16.614577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1736943-f91c-4e0a-a582-35c8633bee5a None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:16.625348 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-a1736943-f91c-4e0a-a582-35c8633bee5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:16.843223 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 860/1702] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:16.848268 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1dd658-de31-4267-a50a-3a7dbc4a802b None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:16.851875 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1b1dd658-de31-4267-a50a-3a7dbc4a802b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:16.923810 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1736943-f91c-4e0a-a582-35c8633bee5a None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:16.941893 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398519c7-86a4-4329-8d9d-f60ff9bf482c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:16.943022 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398519c7-86a4-4329-8d9d-f60ff9bf482c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:16.943304 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398519c7-86a4-4329-8d9d-f60ff9bf482c admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:16.943580 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398519c7-86a4-4329-8d9d-f60ff9bf482c admin admin] PATH_INFO: `/v3/projects/d7a4c63fb6c6476e8f6da76937c9ccd6` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:16.950131 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398519c7-86a4-4329-8d9d-f60ff9bf482c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7a4c63fb6c6476e8f6da76937c9ccd6)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:16.958524 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: WARNING py.warnings [None req-398519c7-86a4-4329-8d9d-f60ff9bf482c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:56:16.958524 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: warnings.warn(msg) May 25 14:56:16.958524 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: May 25 14:56:16.959963 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398519c7-86a4-4329-8d9d-f60ff9bf482c admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:17.007398 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:17.123530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-398519c7-86a4-4329-8d9d-f60ff9bf482c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7a4c63fb6c6476e8f6da76937c9ccd6', 'request_id': 'req-398519c7-86a4-4329-8d9d-f60ff9bf482c'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:17.162242 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-398519c7-86a4-4329-8d9d-f60ff9bf482c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7a4c63fb6c6476e8f6da76937c9ccd6', 'request_id': 'req-398519c7-86a4-4329-8d9d-f60ff9bf482c'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:17.179740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1dd658-de31-4267-a50a-3a7dbc4a802b None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:17.209936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-398519c7-86a4-4329-8d9d-f60ff9bf482c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7a4c63fb6c6476e8f6da76937c9ccd6', 'request_id': 'req-398519c7-86a4-4329-8d9d-f60ff9bf482c'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:17.246596 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-398519c7-86a4-4329-8d9d-f60ff9bf482c admin admin] The token cache is being invalidate because project d7a4c63fb6c6476e8f6da76937c9ccd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:17.246803 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-398519c7-86a4-4329-8d9d-f60ff9bf482c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:17.252028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-398519c7-86a4-4329-8d9d-f60ff9bf482c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:17.255883 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 843/1703] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:56:16 2022] DELETE /identity/v3/projects/d7a4c63fb6c6476e8f6da76937c9ccd6 => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:17.263741 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25fc481-3d75-4637-9f67-b0e27345b433 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:17.264077 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25fc481-3d75-4637-9f67-b0e27345b433 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:17.264286 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25fc481-3d75-4637-9f67-b0e27345b433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:17.337191 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-f25fc481-3d75-4637-9f67-b0e27345b433 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:17.345401 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f25fc481-3d75-4637-9f67-b0e27345b433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:17.414133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28a312-7945-4d40-8e4a-214746550b52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:17.414917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28a312-7945-4d40-8e4a-214746550b52 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:17.415283 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28a312-7945-4d40-8e4a-214746550b52 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:17.415483 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28a312-7945-4d40-8e4a-214746550b52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:17.417640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ab28a312-7945-4d40-8e4a-214746550b52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:17.450453 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:17.577935 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28a312-7945-4d40-8e4a-214746550b52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:17.584834 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28a312-7945-4d40-8e4a-214746550b52 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:17.617823 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 861/1704] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 774 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:17.621799 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28a312-7945-4d40-8e4a-214746550b52 service nova] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:17.628288 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ab28a312-7945-4d40-8e4a-214746550b52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:17.634294 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 844/1705] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:17.640051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25fc481-3d75-4637-9f67-b0e27345b433 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:17.657120 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25fc481-3d75-4637-9f67-b0e27345b433 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:17.677499 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556b63f-ab32-4642-a267-ecb718ff92ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:17.678192 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556b63f-ab32-4642-a267-ecb718ff92ec admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:17.678474 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556b63f-ab32-4642-a267-ecb718ff92ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:17.678770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556b63f-ab32-4642-a267-ecb718ff92ec admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:17.683855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556b63f-ab32-4642-a267-ecb718ff92ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:56:17.684235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556b63f-ab32-4642-a267-ecb718ff92ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:17.691374 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556b63f-ab32-4642-a267-ecb718ff92ec admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:17.727626 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 845/1706] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:56:17.729666 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556b63f-ab32-4642-a267-ecb718ff92ec admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:17.751019 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28a312-7945-4d40-8e4a-214746550b52 service nova] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:17.753967 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556b63f-ab32-4642-a267-ecb718ff92ec admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:17.766531 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973ea2b-57a6-4f22-9573-e228bc84b4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:17.767622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973ea2b-57a6-4f22-9573-e228bc84b4b1 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:17.767622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973ea2b-57a6-4f22-9573-e228bc84b4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:17.767965 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973ea2b-57a6-4f22-9573-e228bc84b4b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:17.769390 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973ea2b-57a6-4f22-9573-e228bc84b4b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:17.771898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973ea2b-57a6-4f22-9573-e228bc84b4b1 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:17.786222 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b0c24-deb2-47fe-93e6-b893d0de4ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:17.787530 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b0c24-deb2-47fe-93e6-b893d0de4ba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:17.787530 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b0c24-deb2-47fe-93e6-b893d0de4ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:17.787715 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b0c24-deb2-47fe-93e6-b893d0de4ba8 admin admin] PATH_INFO: `/v3/users/d921627d3793431eaa7aefd6a3b8db22` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:17.801948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b0c24-deb2-47fe-93e6-b893d0de4ba8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d921627d3793431eaa7aefd6a3b8db22)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:17.804021 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b0c24-deb2-47fe-93e6-b893d0de4ba8 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:17.849641 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 846/1707] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:56:17 2022] POST /identity/v3/projects => generated 403 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:56:17.855164 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973ea2b-57a6-4f22-9573-e228bc84b4b1 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:18.045283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973ea2b-57a6-4f22-9573-e228bc84b4b1 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:18.058718 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f752a5-3759-48b4-9205-4c78e4721e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:18.059416 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f752a5-3759-48b4-9205-4c78e4721e6b admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:18.060362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f752a5-3759-48b4-9205-4c78e4721e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:18.060525 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f752a5-3759-48b4-9205-4c78e4721e6b admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:18.061221 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f752a5-3759-48b4-9205-4c78e4721e6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:18.107260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f752a5-3759-48b4-9205-4c78e4721e6b admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:18.148979 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4d9b0c24-deb2-47fe-93e6-b893d0de4ba8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd921627d3793431eaa7aefd6a3b8db22'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:18.155432 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:18.194144 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 847/1708] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:56:17 2022] POST /identity/v3/users => generated 302 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:56:18.199185 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f752a5-3759-48b4-9205-4c78e4721e6b admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:18.208739 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4d9b0c24-deb2-47fe-93e6-b893d0de4ba8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd921627d3793431eaa7aefd6a3b8db22'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:18.240021 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4d9b0c24-deb2-47fe-93e6-b893d0de4ba8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd921627d3793431eaa7aefd6a3b8db22'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:18.240702 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4d9b0c24-deb2-47fe-93e6-b893d0de4ba8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd921627d3793431eaa7aefd6a3b8db22'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:18.242386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4d9b0c24-deb2-47fe-93e6-b893d0de4ba8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd921627d3793431eaa7aefd6a3b8db22'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:18.255382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f752a5-3759-48b4-9205-4c78e4721e6b admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:18.279952 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e2849-bc8a-4ace-bef4-008c8f158959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:18.281095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e2849-bc8a-4ace-bef4-008c8f158959 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:18.281447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e2849-bc8a-4ace-bef4-008c8f158959 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:18.282156 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e2849-bc8a-4ace-bef4-008c8f158959 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:18.293060 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e2849-bc8a-4ace-bef4-008c8f158959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:18.297412 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e2849-bc8a-4ace-bef4-008c8f158959 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:18.302257 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 862/1709] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:56:17 2022] DELETE /identity/v3/users/d921627d3793431eaa7aefd6a3b8db22 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:18.317141 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 848/1710] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:18.320999 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b0c24-deb2-47fe-93e6-b893d0de4ba8 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:18.324269 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4d9b0c24-deb2-47fe-93e6-b893d0de4ba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:18.421163 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b0c24-deb2-47fe-93e6-b893d0de4ba8 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:18.430335 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8a220-e174-4318-a6c9-183a06ed70d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:18.431128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8a220-e174-4318-a6c9-183a06ed70d9 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:18.431531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8a220-e174-4318-a6c9-183a06ed70d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:18.431957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8a220-e174-4318-a6c9-183a06ed70d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:18.432814 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e8a220-e174-4318-a6c9-183a06ed70d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:18.434909 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e8a220-e174-4318-a6c9-183a06ed70d9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:18.450533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 863/1711] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:18.454554 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e2849-bc8a-4ace-bef4-008c8f158959 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:18.475059 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e2849-bc8a-4ace-bef4-008c8f158959 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:18.480370 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50eb08-51bc-4e64-b165-d3f35df9bd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:18.482804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50eb08-51bc-4e64-b165-d3f35df9bd8f admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:18.483616 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50eb08-51bc-4e64-b165-d3f35df9bd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:18.484265 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50eb08-51bc-4e64-b165-d3f35df9bd8f admin admin] PATH_INFO: `/v3/projects/b634466941414d0f8b36d0c3f1af329d/users/dae1d415876b480b81506aebdb2e5bd7/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:18.511132 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50eb08-51bc-4e64-b165-d3f35df9bd8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b634466941414d0f8b36d0c3f1af329d, user_id=dae1d415876b480b81506aebdb2e5bd7, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:18.518783 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50eb08-51bc-4e64-b165-d3f35df9bd8f admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:18.555186 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 849/1712] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:18 2022] PUT /identity/v3/projects/b634466941414d0f8b36d0c3f1af329d/users/dae1d415876b480b81506aebdb2e5bd7/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:18.565752 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f85e6-2964-486b-bb63-f681c68518c9 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:18.566115 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f85e6-2964-486b-bb63-f681c68518c9 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:18.566668 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f85e6-2964-486b-bb63-f681c68518c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:18.647241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-e04f85e6-2964-486b-bb63-f681c68518c9 None None] MFA Rules not processed for user `dae1d415876b480b81506aebdb2e5bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:18.650854 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e04f85e6-2964-486b-bb63-f681c68518c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:18.694994 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:18.912227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 864/1713] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:18.916378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50eb08-51bc-4e64-b165-d3f35df9bd8f admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:19.305988 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50eb08-51bc-4e64-b165-d3f35df9bd8f admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:19.323119 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c15a5f-39a9-4682-980d-fb7d889bd6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:19.325060 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c15a5f-39a9-4682-980d-fb7d889bd6f0 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:19.325623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c15a5f-39a9-4682-980d-fb7d889bd6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:19.326277 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c15a5f-39a9-4682-980d-fb7d889bd6f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:19.330171 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c15a5f-39a9-4682-980d-fb7d889bd6f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:19.336558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c15a5f-39a9-4682-980d-fb7d889bd6f0 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:19.355164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f85e6-2964-486b-bb63-f681c68518c9 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:19.359917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e04f85e6-2964-486b-bb63-f681c68518c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:19.428573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f85e6-2964-486b-bb63-f681c68518c9 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:19.449790 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29716b44-fb2c-4be4-a542-52a487f953ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:19.453266 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29716b44-fb2c-4be4-a542-52a487f953ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:19.455927 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29716b44-fb2c-4be4-a542-52a487f953ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:19.456248 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29716b44-fb2c-4be4-a542-52a487f953ce admin admin] PATH_INFO: `/v3/projects/2d046f639c4f4aaa9ec3d05168d416f2` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:19.461339 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29716b44-fb2c-4be4-a542-52a487f953ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d046f639c4f4aaa9ec3d05168d416f2)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:19.503055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29716b44-fb2c-4be4-a542-52a487f953ce admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:19.519457 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 850/1714] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:56:18 2022] POST /identity/v3/projects => generated 403 bytes in 606 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:19.521492 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c15a5f-39a9-4682-980d-fb7d889bd6f0 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:19.786746 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-29716b44-fb2c-4be4-a542-52a487f953ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d046f639c4f4aaa9ec3d05168d416f2', 'request_id': 'req-29716b44-fb2c-4be4-a542-52a487f953ce'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:19.820879 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-29716b44-fb2c-4be4-a542-52a487f953ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d046f639c4f4aaa9ec3d05168d416f2', 'request_id': 'req-29716b44-fb2c-4be4-a542-52a487f953ce'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:19.877838 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-29716b44-fb2c-4be4-a542-52a487f953ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d046f639c4f4aaa9ec3d05168d416f2', 'request_id': 'req-29716b44-fb2c-4be4-a542-52a487f953ce'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:19.926455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c15a5f-39a9-4682-980d-fb7d889bd6f0 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:19.982641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-29716b44-fb2c-4be4-a542-52a487f953ce admin admin] The token cache is being invalidate because project 2d046f639c4f4aaa9ec3d05168d416f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:19.982966 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-29716b44-fb2c-4be4-a542-52a487f953ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:19.998604 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-29716b44-fb2c-4be4-a542-52a487f953ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:20.008567 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 865/1715] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:56:19 2022] DELETE /identity/v3/projects/2d046f639c4f4aaa9ec3d05168d416f2 => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:20.014998 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29716b44-fb2c-4be4-a542-52a487f953ce admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:20.017100 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-29716b44-fb2c-4be4-a542-52a487f953ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:20.426328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29716b44-fb2c-4be4-a542-52a487f953ce admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:20.442323 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579888f7-5130-4ab8-b93c-2668e3be5c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:20.445932 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579888f7-5130-4ab8-b93c-2668e3be5c53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:20.446455 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579888f7-5130-4ab8-b93c-2668e3be5c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:20.449509 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579888f7-5130-4ab8-b93c-2668e3be5c53 admin admin] PATH_INFO: `/v3/users/fe05ae55efb646b88211609a47870eed` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:20.451755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579888f7-5130-4ab8-b93c-2668e3be5c53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe05ae55efb646b88211609a47870eed)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:20.456739 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579888f7-5130-4ab8-b93c-2668e3be5c53 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:20.622657 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:20.651478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-579888f7-5130-4ab8-b93c-2668e3be5c53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe05ae55efb646b88211609a47870eed'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:20.705677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-579888f7-5130-4ab8-b93c-2668e3be5c53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe05ae55efb646b88211609a47870eed'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:20.713591 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:20.742345 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d741d-9e1d-412f-aeac-6e47f7360956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:20.745298 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d741d-9e1d-412f-aeac-6e47f7360956 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:20.745298 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d741d-9e1d-412f-aeac-6e47f7360956 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:20.746539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d741d-9e1d-412f-aeac-6e47f7360956 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:20.749513 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d741d-9e1d-412f-aeac-6e47f7360956 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:20.755042 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-579888f7-5130-4ab8-b93c-2668e3be5c53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe05ae55efb646b88211609a47870eed'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:20.756440 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d741d-9e1d-412f-aeac-6e47f7360956 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:20.756803 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-579888f7-5130-4ab8-b93c-2668e3be5c53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe05ae55efb646b88211609a47870eed'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:20.757785 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-579888f7-5130-4ab8-b93c-2668e3be5c53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe05ae55efb646b88211609a47870eed'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:20.818741 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 851/1716] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:56:19 2022] POST /identity/v3/users => generated 308 bytes in 1299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:20.825247 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d741d-9e1d-412f-aeac-6e47f7360956 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:20.828556 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-778d741d-9e1d-412f-aeac-6e47f7360956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:20.833719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 866/1717] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:56:20 2022] DELETE /identity/v3/users/fe05ae55efb646b88211609a47870eed => generated 0 bytes in 817 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:20.839394 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea74000-39f8-4f9a-ade0-d545c3d872d8 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:20.844632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea74000-39f8-4f9a-ade0-d545c3d872d8 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:20.844839 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea74000-39f8-4f9a-ade0-d545c3d872d8 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:20.848141 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 867/1718] 10.222.0.78 () {54 vars in 912 bytes} [Wed May 25 14:56:20 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:56:20.852613 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea74000-39f8-4f9a-ade0-d545c3d872d8 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:20.864773 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3ea74000-39f8-4f9a-ade0-d545c3d872d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:20.880170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d741d-9e1d-412f-aeac-6e47f7360956 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:21.232103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc6ad4-cc15-438b-a360-5b78c272e0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:21.239750 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc6ad4-cc15-438b-a360-5b78c272e0ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:21.241224 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc6ad4-cc15-438b-a360-5b78c272e0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:21.243501 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc6ad4-cc15-438b-a360-5b78c272e0ee admin admin] PATH_INFO: `/v3/projects/5bbe9f9991bf4f9b876015d1d146b066` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:21.252559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc6ad4-cc15-438b-a360-5b78c272e0ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bbe9f9991bf4f9b876015d1d146b066)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:21.265867 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc6ad4-cc15-438b-a360-5b78c272e0ee admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:21.339702 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea74000-39f8-4f9a-ade0-d545c3d872d8 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:21.353883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b7733-b067-4561-b659-cb599ab7458c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:21.359051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b7733-b067-4561-b659-cb599ab7458c admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:21.359308 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b7733-b067-4561-b659-cb599ab7458c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:21.362667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b7733-b067-4561-b659-cb599ab7458c admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:21.367227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b7733-b067-4561-b659-cb599ab7458c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:21.376176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b7733-b067-4561-b659-cb599ab7458c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:21.461545 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 868/1719] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 613 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:21.480553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d4bd0-209c-4c83-a124-ea828cd85360 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:21.481194 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d4bd0-209c-4c83-a124-ea828cd85360 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:21.482318 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d4bd0-209c-4c83-a124-ea828cd85360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:21.590252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b9cc6ad4-cc15-438b-a360-5b78c272e0ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bbe9f9991bf4f9b876015d1d146b066', 'request_id': 'req-b9cc6ad4-cc15-438b-a360-5b78c272e0ee'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:21.601650 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-151d4bd0-209c-4c83-a124-ea828cd85360 None None] MFA Rules not processed for user `70ad51a8ad68440da6618891e2e08cb8`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:21.610304 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-151d4bd0-209c-4c83-a124-ea828cd85360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:21.615858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b9cc6ad4-cc15-438b-a360-5b78c272e0ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bbe9f9991bf4f9b876015d1d146b066', 'request_id': 'req-b9cc6ad4-cc15-438b-a360-5b78c272e0ee'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:21.640253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b9cc6ad4-cc15-438b-a360-5b78c272e0ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bbe9f9991bf4f9b876015d1d146b066', 'request_id': 'req-b9cc6ad4-cc15-438b-a360-5b78c272e0ee'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:21.682363 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b9cc6ad4-cc15-438b-a360-5b78c272e0ee admin admin] The token cache is being invalidate because project 5bbe9f9991bf4f9b876015d1d146b066 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:21.682675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b9cc6ad4-cc15-438b-a360-5b78c272e0ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:21.684351 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b9cc6ad4-cc15-438b-a360-5b78c272e0ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:21.686389 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 852/1720] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:56:20 2022] DELETE /identity/v3/projects/5bbe9f9991bf4f9b876015d1d146b066 => generated 0 bytes in 866 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:21.690212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc6ad4-cc15-438b-a360-5b78c272e0ee admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:21.693179 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-b9cc6ad4-cc15-438b-a360-5b78c272e0ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:21.789653 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 869/1721] 10.222.0.78 () {58 vars in 1009 bytes} [Wed May 25 14:56:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:21.792205 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dedb06-53aa-40b1-a514-3125715391d2 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:21.792730 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dedb06-53aa-40b1-a514-3125715391d2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:21.793023 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dedb06-53aa-40b1-a514-3125715391d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:21.887613 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-f0dedb06-53aa-40b1-a514-3125715391d2 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:21.889288 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f0dedb06-53aa-40b1-a514-3125715391d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:21.971288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc6ad4-cc15-438b-a360-5b78c272e0ee admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:22.063670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4563343-d61a-4c54-b195-3fbc81f4b14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:22.064980 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4563343-d61a-4c54-b195-3fbc81f4b14c admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:22.065478 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4563343-d61a-4c54-b195-3fbc81f4b14c admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:22.065812 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4563343-d61a-4c54-b195-3fbc81f4b14c admin admin] PATH_INFO: `/v3/projects/806c800509b340a0b4768da6251fb7a0/users/e72ccf33ec83465398b81653fa81b57c/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:22.124813 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4563343-d61a-4c54-b195-3fbc81f4b14c admin admin] RBAC: Authorizing `identity:create_grant(project_id=806c800509b340a0b4768da6251fb7a0, user_id=e72ccf33ec83465398b81653fa81b57c, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:22.125756 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4563343-d61a-4c54-b195-3fbc81f4b14c admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:22.131304 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:22.160109 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 853/1722] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:21 2022] PUT /identity/v3/projects/806c800509b340a0b4768da6251fb7a0/users/e72ccf33ec83465398b81653fa81b57c/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:22.161215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4563343-d61a-4c54-b195-3fbc81f4b14c admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:22.165005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-a4563343-d61a-4c54-b195-3fbc81f4b14c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:22.296438 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4563343-d61a-4c54-b195-3fbc81f4b14c admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:22.309295 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 870/1723] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 519 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:22.311224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dedb06-53aa-40b1-a514-3125715391d2 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:22.311498 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8d0d2-54b9-425c-a6cf-8b32e51c3f6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:22.312480 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8d0d2-54b9-425c-a6cf-8b32e51c3f6f service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:22.312874 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8d0d2-54b9-425c-a6cf-8b32e51c3f6f service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:22.313258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8d0d2-54b9-425c-a6cf-8b32e51c3f6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:22.327714 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:22.334571 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba8d0d2-54b9-425c-a6cf-8b32e51c3f6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:22.336627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba8d0d2-54b9-425c-a6cf-8b32e51c3f6f service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:22.341874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dedb06-53aa-40b1-a514-3125715391d2 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:22.376106 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 854/1724] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:22.378645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c922a5-6ece-449e-8558-69831e5a3e81 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:22.380704 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c922a5-6ece-449e-8558-69831e5a3e81 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:22.381719 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c922a5-6ece-449e-8558-69831e5a3e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:22.456660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-b1c922a5-6ece-449e-8558-69831e5a3e81 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:22.460890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-b1c922a5-6ece-449e-8558-69831e5a3e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:22.655990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd684b3-2852-4d13-92ef-92af34374068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:22.656925 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd684b3-2852-4d13-92ef-92af34374068 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:22.656925 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd684b3-2852-4d13-92ef-92af34374068 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:22.656925 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd684b3-2852-4d13-92ef-92af34374068 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:22.665988 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd684b3-2852-4d13-92ef-92af34374068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:22.668846 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd684b3-2852-4d13-92ef-92af34374068 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:22.708051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 871/1725] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:22.715106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd684b3-2852-4d13-92ef-92af34374068 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:22.761734 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd684b3-2852-4d13-92ef-92af34374068 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:22.787481 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece9bc7-6243-4e00-8cac-316c9da77e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:22.788583 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece9bc7-6243-4e00-8cac-316c9da77e1e admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:22.788875 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece9bc7-6243-4e00-8cac-316c9da77e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:22.789160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece9bc7-6243-4e00-8cac-316c9da77e1e admin admin] PATH_INFO: `/v3/projects/806c800509b340a0b4768da6251fb7a0/users/e72ccf33ec83465398b81653fa81b57c/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:22.791373 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece9bc7-6243-4e00-8cac-316c9da77e1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=806c800509b340a0b4768da6251fb7a0, user_id=e72ccf33ec83465398b81653fa81b57c, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:22.796094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece9bc7-6243-4e00-8cac-316c9da77e1e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:22.852020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 855/1727] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 475 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:22.854317 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 872/1727] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:22 2022] PUT /identity/v3/projects/806c800509b340a0b4768da6251fb7a0/users/e72ccf33ec83465398b81653fa81b57c/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:22.855208 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c922a5-6ece-449e-8558-69831e5a3e81 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:22.855948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92dfc6-8ebe-4a4a-9b2f-cc8ed3368f93 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:22.858408 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92dfc6-8ebe-4a4a-9b2f-cc8ed3368f93 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:22.858750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92dfc6-8ebe-4a4a-9b2f-cc8ed3368f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:22.890122 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c922a5-6ece-449e-8558-69831e5a3e81 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:22.969107 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-8a92dfc6-8ebe-4a4a-9b2f-cc8ed3368f93 None None] MFA Rules not processed for user `e72ccf33ec83465398b81653fa81b57c`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:22.971731 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8a92dfc6-8ebe-4a4a-9b2f-cc8ed3368f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:23.054776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa6820-e36e-4a12-9c91-66cae170eb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:23.055470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa6820-e36e-4a12-9c91-66cae170eb7a admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:23.055751 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa6820-e36e-4a12-9c91-66cae170eb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:23.056296 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa6820-e36e-4a12-9c91-66cae170eb7a admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:23.057780 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa6820-e36e-4a12-9c91-66cae170eb7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:56:23.062330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa6820-e36e-4a12-9c91-66cae170eb7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:23.066836 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa6820-e36e-4a12-9c91-66cae170eb7a admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:23.147056 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 856/1728] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:56:23.152232 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42433694-4d15-4f85-8e6d-d75a06aa1eff None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:23.152885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42433694-4d15-4f85-8e6d-d75a06aa1eff None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:23.156970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42433694-4d15-4f85-8e6d-d75a06aa1eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:23.209437 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-42433694-4d15-4f85-8e6d-d75a06aa1eff None None] MFA Rules not processed for user `7fedb9b92e9a4e3cb671f69751e426a1`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:23.213253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-42433694-4d15-4f85-8e6d-d75a06aa1eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:23.275955 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 873/1729] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:22 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:23.279426 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92dfc6-8ebe-4a4a-9b2f-cc8ed3368f93 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:23.351753 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92dfc6-8ebe-4a4a-9b2f-cc8ed3368f93 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:23.359653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:23.372080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa38fc-1c6f-4bc3-a6e5-d3059a54432c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:23.375626 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa38fc-1c6f-4bc3-a6e5-d3059a54432c admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:23.376936 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa38fc-1c6f-4bc3-a6e5-d3059a54432c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:23.377407 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa38fc-1c6f-4bc3-a6e5-d3059a54432c admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:23.380980 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa38fc-1c6f-4bc3-a6e5-d3059a54432c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:23.391601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa38fc-1c6f-4bc3-a6e5-d3059a54432c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:23.461978 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 874/1730] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:56:23 2022] POST /identity/v3/projects => generated 365 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:23.463603 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa38fc-1c6f-4bc3-a6e5-d3059a54432c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:23.470087 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9afa38fc-1c6f-4bc3-a6e5-d3059a54432c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:23.489024 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 857/1731] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:23.490334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42433694-4d15-4f85-8e6d-d75a06aa1eff None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:23.545092 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42433694-4d15-4f85-8e6d-d75a06aa1eff None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:23.736305 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa38fc-1c6f-4bc3-a6e5-d3059a54432c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:23.756421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d14c54-ef53-4b7b-9ea4-c7134a5fa146 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:23.759502 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d14c54-ef53-4b7b-9ea4-c7134a5fa146 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:23.760155 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d14c54-ef53-4b7b-9ea4-c7134a5fa146 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:23.760155 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d14c54-ef53-4b7b-9ea4-c7134a5fa146 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:23.785613 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:23.786729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09cd8e-2999-48cf-86f5-e2b0bb385c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:23.787795 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09cd8e-2999-48cf-86f5-e2b0bb385c1b admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:23.788225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09cd8e-2999-48cf-86f5-e2b0bb385c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:23.788452 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09cd8e-2999-48cf-86f5-e2b0bb385c1b admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:23.789441 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c09cd8e-2999-48cf-86f5-e2b0bb385c1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:23.807549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c09cd8e-2999-48cf-86f5-e2b0bb385c1b admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:23.867331 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 858/1732] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:56:23 2022] POST /identity/v3/users => generated 283 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:23.873109 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09cd8e-2999-48cf-86f5-e2b0bb385c1b admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:23.941759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09cd8e-2999-48cf-86f5-e2b0bb385c1b admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:23.977912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd94e34-1df4-43e0-9488-cb8e319e006e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:23.979062 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd94e34-1df4-43e0-9488-cb8e319e006e admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:23.979542 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd94e34-1df4-43e0-9488-cb8e319e006e admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:23.980449 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd94e34-1df4-43e0-9488-cb8e319e006e admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:23.987355 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd94e34-1df4-43e0-9488-cb8e319e006e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:23.989617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd94e34-1df4-43e0-9488-cb8e319e006e admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:24.047338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 859/1733] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:56:24.056087 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d14c54-ef53-4b7b-9ea4-c7134a5fa146 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:24.058117 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d14c54-ef53-4b7b-9ea4-c7134a5fa146 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:24.062932 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd94e34-1df4-43e0-9488-cb8e319e006e admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:24.085924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-3dd94e34-1df4-43e0-9488-cb8e319e006e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:24.120252 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 875/1734] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:23 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:24.122625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d14c54-ef53-4b7b-9ea4-c7134a5fa146 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:24.179652 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd94e34-1df4-43e0-9488-cb8e319e006e admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:24.197109 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d14c54-ef53-4b7b-9ea4-c7134a5fa146 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:24.215763 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af418352-41bd-4446-8c0d-b0d456a5d7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:24.216997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af418352-41bd-4446-8c0d-b0d456a5d7a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:24.217282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af418352-41bd-4446-8c0d-b0d456a5d7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:24.217778 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af418352-41bd-4446-8c0d-b0d456a5d7a8 admin admin] PATH_INFO: `/v3/projects/bde86e084d2a4e0caf9ac518a477ad09` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:24.223064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af418352-41bd-4446-8c0d-b0d456a5d7a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bde86e084d2a4e0caf9ac518a477ad09)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:24.225530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af418352-41bd-4446-8c0d-b0d456a5d7a8 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:24.233984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29e0e2-9921-4844-8428-2f964c1733cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:24.237988 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29e0e2-9921-4844-8428-2f964c1733cb admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:24.238312 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29e0e2-9921-4844-8428-2f964c1733cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:24.238592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29e0e2-9921-4844-8428-2f964c1733cb admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:24.239368 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b29e0e2-9921-4844-8428-2f964c1733cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:24.249764 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b29e0e2-9921-4844-8428-2f964c1733cb admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:24.292819 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 876/1735] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:24.294609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29e0e2-9921-4844-8428-2f964c1733cb admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:24.323210 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29e0e2-9921-4844-8428-2f964c1733cb admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:24.341097 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a489b-15fd-4578-aaa9-4a55aa0e1cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:24.342093 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a489b-15fd-4578-aaa9-4a55aa0e1cfd admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:24.342334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a489b-15fd-4578-aaa9-4a55aa0e1cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:24.342612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a489b-15fd-4578-aaa9-4a55aa0e1cfd admin admin] PATH_INFO: `/v3/projects/3c9b886ab2b54d9a9751f08adda65517/users/b54594f051334e0282f146c626064717/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:24.400339 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a489b-15fd-4578-aaa9-4a55aa0e1cfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c9b886ab2b54d9a9751f08adda65517, user_id=b54594f051334e0282f146c626064717, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:24.405539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a489b-15fd-4578-aaa9-4a55aa0e1cfd admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:24.416051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-af418352-41bd-4446-8c0d-b0d456a5d7a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bde86e084d2a4e0caf9ac518a477ad09', 'request_id': 'req-af418352-41bd-4446-8c0d-b0d456a5d7a8'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:24.430111 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-af418352-41bd-4446-8c0d-b0d456a5d7a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bde86e084d2a4e0caf9ac518a477ad09', 'request_id': 'req-af418352-41bd-4446-8c0d-b0d456a5d7a8'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:24.453000 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 877/1736] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:24 2022] PUT /identity/v3/projects/3c9b886ab2b54d9a9751f08adda65517/users/b54594f051334e0282f146c626064717/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:24.463737 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15355a37-317b-4729-a05e-82bf2b4e4962 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:24.465455 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15355a37-317b-4729-a05e-82bf2b4e4962 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:24.465892 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15355a37-317b-4729-a05e-82bf2b4e4962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:24.483476 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-af418352-41bd-4446-8c0d-b0d456a5d7a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bde86e084d2a4e0caf9ac518a477ad09', 'request_id': 'req-af418352-41bd-4446-8c0d-b0d456a5d7a8'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:24.535658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-15355a37-317b-4729-a05e-82bf2b4e4962 None None] MFA Rules not processed for user `b54594f051334e0282f146c626064717`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:24.539315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-15355a37-317b-4729-a05e-82bf2b4e4962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:24.552469 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-af418352-41bd-4446-8c0d-b0d456a5d7a8 admin admin] The token cache is being invalidate because project bde86e084d2a4e0caf9ac518a477ad09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:24.553354 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-af418352-41bd-4446-8c0d-b0d456a5d7a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:24.562847 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-af418352-41bd-4446-8c0d-b0d456a5d7a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:24.565498 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 860/1737] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:56:24 2022] DELETE /identity/v3/projects/bde86e084d2a4e0caf9ac518a477ad09 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:24.572667 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3098653-d96b-4c1b-92c1-3dac0cc46e55 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:24.572667 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3098653-d96b-4c1b-92c1-3dac0cc46e55 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:24.572667 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3098653-d96b-4c1b-92c1-3dac0cc46e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:24.646669 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-c3098653-d96b-4c1b-92c1-3dac0cc46e55 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:24.652239 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-c3098653-d96b-4c1b-92c1-3dac0cc46e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:24.764344 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 878/1738] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:24.768416 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15355a37-317b-4729-a05e-82bf2b4e4962 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:24.772527 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-15355a37-317b-4729-a05e-82bf2b4e4962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:24.820151 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:24.972378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 861/1739] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:56:24.974741 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3098653-d96b-4c1b-92c1-3dac0cc46e55 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:24.996119 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-c3098653-d96b-4c1b-92c1-3dac0cc46e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:25.049548 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3098653-d96b-4c1b-92c1-3dac0cc46e55 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:25.071122 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5df92-98e6-45e3-acad-19ebfc075c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:25.073386 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5df92-98e6-45e3-acad-19ebfc075c01 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:25.075365 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5df92-98e6-45e3-acad-19ebfc075c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:25.075799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5df92-98e6-45e3-acad-19ebfc075c01 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:25.076665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5df92-98e6-45e3-acad-19ebfc075c01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:25.088619 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5df92-98e6-45e3-acad-19ebfc075c01 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:25.193531 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 862/1740] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:56:24 2022] POST /identity/v3/projects => generated 365 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:25.196658 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221bef8-d912-404c-a3ba-65b9889764c1 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:25.196658 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221bef8-d912-404c-a3ba-65b9889764c1 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:25.201424 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221bef8-d912-404c-a3ba-65b9889764c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:25.206464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:25.255885 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15355a37-317b-4729-a05e-82bf2b4e4962 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:25.281243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-2221bef8-d912-404c-a3ba-65b9889764c1 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:25.282951 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-2221bef8-d912-404c-a3ba-65b9889764c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:25.498656 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f02ef0-0f31-4ce3-bba8-85ed2903e534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:25.499630 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f02ef0-0f31-4ce3-bba8-85ed2903e534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:25.500219 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f02ef0-0f31-4ce3-bba8-85ed2903e534 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:25.500219 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f02ef0-0f31-4ce3-bba8-85ed2903e534 admin admin] PATH_INFO: `/v3/users/dae1d415876b480b81506aebdb2e5bd7` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:25.501467 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f02ef0-0f31-4ce3-bba8-85ed2903e534 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dae1d415876b480b81506aebdb2e5bd7)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:25.503490 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f02ef0-0f31-4ce3-bba8-85ed2903e534 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:25.607322 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 863/1741] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:25.610847 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da97db-c97e-4c5f-93b6-5f4c79e3da30 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:25.617138 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da97db-c97e-4c5f-93b6-5f4c79e3da30 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:25.620411 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da97db-c97e-4c5f-93b6-5f4c79e3da30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:25.664803 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d7f02ef0-0f31-4ce3-bba8-85ed2903e534 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dae1d415876b480b81506aebdb2e5bd7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:25.725386 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-d8da97db-c97e-4c5f-93b6-5f4c79e3da30 None None] MFA Rules not processed for user `70ad51a8ad68440da6618891e2e08cb8`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:25.727400 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d7f02ef0-0f31-4ce3-bba8-85ed2903e534 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dae1d415876b480b81506aebdb2e5bd7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:25.728400 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d8da97db-c97e-4c5f-93b6-5f4c79e3da30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:25.747291 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d7f02ef0-0f31-4ce3-bba8-85ed2903e534 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dae1d415876b480b81506aebdb2e5bd7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:25.750054 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d7f02ef0-0f31-4ce3-bba8-85ed2903e534 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dae1d415876b480b81506aebdb2e5bd7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:25.756479 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d7f02ef0-0f31-4ce3-bba8-85ed2903e534 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dae1d415876b480b81506aebdb2e5bd7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:25.817125 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 879/1742] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:56:24 2022] DELETE /identity/v3/users/dae1d415876b480b81506aebdb2e5bd7 => generated 0 bytes in 1052 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:25.818334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f02ef0-0f31-4ce3-bba8-85ed2903e534 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:25.820801 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d7f02ef0-0f31-4ce3-bba8-85ed2903e534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:25.975299 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 864/1743] 10.222.0.72 () {58 vars in 1015 bytes} [Wed May 25 14:56:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:25.978045 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbd4c4-f54c-460f-a6a1-324c20733d08 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:25.978344 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbd4c4-f54c-460f-a6a1-324c20733d08 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:25.978637 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbd4c4-f54c-460f-a6a1-324c20733d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:26.050598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-cfcbd4c4-f54c-460f-a6a1-324c20733d08 None None] MFA Rules not processed for user `e60a5e751f22450c8e366c85b37af1f6`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:26.051810 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-cfcbd4c4-f54c-460f-a6a1-324c20733d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:26.130411 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f02ef0-0f31-4ce3-bba8-85ed2903e534 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:26.137921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb3305-4dbb-44fc-913d-72fad9672acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:26.139070 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb3305-4dbb-44fc-913d-72fad9672acf admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:26.139500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb3305-4dbb-44fc-913d-72fad9672acf admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:26.140024 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb3305-4dbb-44fc-913d-72fad9672acf admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:26.141791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb3305-4dbb-44fc-913d-72fad9672acf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:26.152158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb3305-4dbb-44fc-913d-72fad9672acf admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:26.212377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 880/1744] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:56:25 2022] POST /identity/v3/users => generated 289 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:26.215199 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb3305-4dbb-44fc-913d-72fad9672acf admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:26.225669 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-07cb3305-4dbb-44fc-913d-72fad9672acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:26.234702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:26.289403 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 865/1745] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:26.292699 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbd4c4-f54c-460f-a6a1-324c20733d08 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:26.294567 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb3305-4dbb-44fc-913d-72fad9672acf admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:26.296909 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-cfcbd4c4-f54c-460f-a6a1-324c20733d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:26.315327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152e3f1-5e13-4176-88a3-4321b914196d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:26.316108 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152e3f1-5e13-4176-88a3-4321b914196d admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:26.322766 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152e3f1-5e13-4176-88a3-4321b914196d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:26.323112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152e3f1-5e13-4176-88a3-4321b914196d admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:26.323875 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1152e3f1-5e13-4176-88a3-4321b914196d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:56:26.324231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1152e3f1-5e13-4176-88a3-4321b914196d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:26.334267 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1152e3f1-5e13-4176-88a3-4321b914196d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:26.357608 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbd4c4-f54c-460f-a6a1-324c20733d08 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:26.362693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:26.385046 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 881/1746] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:56:26.385046 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d1682-5011-4397-8e24-6f78fd5acb7c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:26.385046 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d1682-5011-4397-8e24-6f78fd5acb7c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:26.385046 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d1682-5011-4397-8e24-6f78fd5acb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:26.386225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7799cacb-27f9-4d5d-b453-a3745dfe4554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:26.386561 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799cacb-27f9-4d5d-b453-a3745dfe4554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:26.386844 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799cacb-27f9-4d5d-b453-a3745dfe4554 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:26.387116 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799cacb-27f9-4d5d-b453-a3745dfe4554 admin admin] PATH_INFO: `/v3/users/3393d62d88f74e80901615e71d8cbffb` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:26.395182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7799cacb-27f9-4d5d-b453-a3745dfe4554 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3393d62d88f74e80901615e71d8cbffb)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:26.397257 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7799cacb-27f9-4d5d-b453-a3745dfe4554 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:26.455134 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-507d1682-5011-4397-8e24-6f78fd5acb7c None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:26.455853 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-507d1682-5011-4397-8e24-6f78fd5acb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:26.492001 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 882/1747] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:26.493016 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d1682-5011-4397-8e24-6f78fd5acb7c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:26.504630 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-7799cacb-27f9-4d5d-b453-a3745dfe4554 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3393d62d88f74e80901615e71d8cbffb'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:26.523202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d1682-5011-4397-8e24-6f78fd5acb7c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:26.541447 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e51a4a-6e98-40c1-9252-ed72ae040bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:26.542662 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e51a4a-6e98-40c1-9252-ed72ae040bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:26.542662 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e51a4a-6e98-40c1-9252-ed72ae040bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:26.542843 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e51a4a-6e98-40c1-9252-ed72ae040bf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:26.543759 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e51a4a-6e98-40c1-9252-ed72ae040bf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:26.545447 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e51a4a-6e98-40c1-9252-ed72ae040bf9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:26.556696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-7799cacb-27f9-4d5d-b453-a3745dfe4554 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3393d62d88f74e80901615e71d8cbffb'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:26.581630 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-7799cacb-27f9-4d5d-b453-a3745dfe4554 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3393d62d88f74e80901615e71d8cbffb'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:26.582812 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-7799cacb-27f9-4d5d-b453-a3745dfe4554 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3393d62d88f74e80901615e71d8cbffb'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:26.583877 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-7799cacb-27f9-4d5d-b453-a3745dfe4554 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3393d62d88f74e80901615e71d8cbffb'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:26.584847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 883/1748] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:26.585719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e51a4a-6e98-40c1-9252-ed72ae040bf9 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:26.607030 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-06e51a4a-6e98-40c1-9252-ed72ae040bf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:26.653887 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 866/1749] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:56:26 2022] DELETE /identity/v3/users/3393d62d88f74e80901615e71d8cbffb => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:26.657579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7799cacb-27f9-4d5d-b453-a3745dfe4554 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:26.664746 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-7799cacb-27f9-4d5d-b453-a3745dfe4554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:26.679450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e51a4a-6e98-40c1-9252-ed72ae040bf9 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:26.868122 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e2c60-4eec-4772-a9f7-44d3496be892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:26.869480 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e2c60-4eec-4772-a9f7-44d3496be892 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:26.870578 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e2c60-4eec-4772-a9f7-44d3496be892 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:26.871645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7799cacb-27f9-4d5d-b453-a3745dfe4554 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:26.872494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e2c60-4eec-4772-a9f7-44d3496be892 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:26.875571 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e2c60-4eec-4772-a9f7-44d3496be892 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:26.880843 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e2c60-4eec-4772-a9f7-44d3496be892 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:26.881043 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e185058-559e-4292-9c9d-c10cc9dc7fef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:26.884060 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e185058-559e-4292-9c9d-c10cc9dc7fef service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:26.884520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e185058-559e-4292-9c9d-c10cc9dc7fef service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:26.884913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e185058-559e-4292-9c9d-c10cc9dc7fef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:26.887354 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-6e185058-559e-4292-9c9d-c10cc9dc7fef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:26.911898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e185058-559e-4292-9c9d-c10cc9dc7fef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:26.918550 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e185058-559e-4292-9c9d-c10cc9dc7fef service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:26.941560 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 867/1750] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:26.943195 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e185058-559e-4292-9c9d-c10cc9dc7fef service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:26.947401 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-6e185058-559e-4292-9c9d-c10cc9dc7fef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:26.961462 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e185058-559e-4292-9c9d-c10cc9dc7fef service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:26.978771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586bf7fa-6ffd-4b8f-84d4-7d12f498c674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:26.979826 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bf7fa-6ffd-4b8f-84d4-7d12f498c674 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:26.980487 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bf7fa-6ffd-4b8f-84d4-7d12f498c674 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:26.982529 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bf7fa-6ffd-4b8f-84d4-7d12f498c674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:26.984231 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-586bf7fa-6ffd-4b8f-84d4-7d12f498c674 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:27.001170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586bf7fa-6ffd-4b8f-84d4-7d12f498c674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:27.003301 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586bf7fa-6ffd-4b8f-84d4-7d12f498c674 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:27.017612 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 868/1751] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:27.031392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96291e2-372e-4b27-8caa-ff81a132d2c8 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:27.032900 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96291e2-372e-4b27-8caa-ff81a132d2c8 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:27.033336 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96291e2-372e-4b27-8caa-ff81a132d2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:27.033887 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 884/1752] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:56:26 2022] POST /identity/v3/projects => generated 365 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:27.034796 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e2c60-4eec-4772-a9f7-44d3496be892 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:27.046408 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4b0e2c60-4eec-4772-a9f7-44d3496be892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:27.093098 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-d96291e2-372e-4b27-8caa-ff81a132d2c8 None None] MFA Rules not processed for user `97011bf8f6d444468fecdde7acfa1020`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:27.096190 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d96291e2-372e-4b27-8caa-ff81a132d2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:27.287135 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 869/1753] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:27.288669 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e8ee2-6586-4f16-b776-e8daae6a155a None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:27.289009 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e8ee2-6586-4f16-b776-e8daae6a155a None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:27.289382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e8ee2-6586-4f16-b776-e8daae6a155a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:27.297626 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:27.314367 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-5b0e8ee2-6586-4f16-b776-e8daae6a155a None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:27.316212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-5b0e8ee2-6586-4f16-b776-e8daae6a155a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:27.404894 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e2c60-4eec-4772-a9f7-44d3496be892 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:27.428758 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9f614-088b-4154-b640-a20f8199cd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:27.430399 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9f614-088b-4154-b640-a20f8199cd33 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:27.430700 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9f614-088b-4154-b640-a20f8199cd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:27.431075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9f614-088b-4154-b640-a20f8199cd33 admin admin] PATH_INFO: `/v3/projects/8a73b10aa92d452a890c7d906b8303c3/users/c1623f961a8b4002a26c50a14cdb85da/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:27.478685 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c9f614-088b-4154-b640-a20f8199cd33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a73b10aa92d452a890c7d906b8303c3, user_id=c1623f961a8b4002a26c50a14cdb85da, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:27.482645 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c9f614-088b-4154-b640-a20f8199cd33 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:27.519888 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 885/1754] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:27 2022] PUT /identity/v3/projects/8a73b10aa92d452a890c7d906b8303c3/users/c1623f961a8b4002a26c50a14cdb85da/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:27.520828 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9f614-088b-4154-b640-a20f8199cd33 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:27.525610 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-21c9f614-088b-4154-b640-a20f8199cd33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:27.526770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 870/1755] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:27.546842 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e8ee2-6586-4f16-b776-e8daae6a155a None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:27.605259 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e8ee2-6586-4f16-b776-e8daae6a155a None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:27.679354 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:27.748486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9f614-088b-4154-b640-a20f8199cd33 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:27.768476 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12af02-c70a-4679-9d9e-6495ef0f63b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:27.771504 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12af02-c70a-4679-9d9e-6495ef0f63b7 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:27.771824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12af02-c70a-4679-9d9e-6495ef0f63b7 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:27.772864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12af02-c70a-4679-9d9e-6495ef0f63b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:27.780272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-af12af02-c70a-4679-9d9e-6495ef0f63b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:27.821198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762be0d-b518-497a-8536-70915559c4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:27.823358 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762be0d-b518-497a-8536-70915559c4ca admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:27.823942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762be0d-b518-497a-8536-70915559c4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:27.825302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762be0d-b518-497a-8536-70915559c4ca admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:27.828206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d762be0d-b518-497a-8536-70915559c4ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:27.830389 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d762be0d-b518-497a-8536-70915559c4ca admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:27.881618 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 871/1756] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:56:27 2022] POST /identity/v3/users => generated 283 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:27.882838 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762be0d-b518-497a-8536-70915559c4ca admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:27.916184 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762be0d-b518-497a-8536-70915559c4ca admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:27.930690 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39c1ba-2ecf-48c9-96d5-b224839f54d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:27.932833 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39c1ba-2ecf-48c9-96d5-b224839f54d8 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:27.933207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39c1ba-2ecf-48c9-96d5-b224839f54d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:27.934703 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39c1ba-2ecf-48c9-96d5-b224839f54d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:27.935519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39c1ba-2ecf-48c9-96d5-b224839f54d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:27.939367 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39c1ba-2ecf-48c9-96d5-b224839f54d8 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:27.955540 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12af02-c70a-4679-9d9e-6495ef0f63b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:27.957430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12af02-c70a-4679-9d9e-6495ef0f63b7 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:27.974290 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 872/1757] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:27.976166 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39c1ba-2ecf-48c9-96d5-b224839f54d8 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:27.976888 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 886/1758] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:27.977734 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12af02-c70a-4679-9d9e-6495ef0f63b7 service nova] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:27.992555 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12af02-c70a-4679-9d9e-6495ef0f63b7 service nova] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:28.000232 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337db8b-f69e-4944-9062-fe88a6f97281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:28.001551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337db8b-f69e-4944-9062-fe88a6f97281 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:28.001884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337db8b-f69e-4944-9062-fe88a6f97281 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:28.002418 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337db8b-f69e-4944-9062-fe88a6f97281 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:28.003190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337db8b-f69e-4944-9062-fe88a6f97281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:28.005494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337db8b-f69e-4944-9062-fe88a6f97281 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:28.032404 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 887/1759] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:28.033619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337db8b-f69e-4944-9062-fe88a6f97281 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:28.060372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337db8b-f69e-4944-9062-fe88a6f97281 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:28.075924 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7eedf1-371e-473b-9ec2-9b9afb4c5d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:28.077203 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7eedf1-371e-473b-9ec2-9b9afb4c5d1e admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:28.077429 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7eedf1-371e-473b-9ec2-9b9afb4c5d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:28.079315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7eedf1-371e-473b-9ec2-9b9afb4c5d1e admin admin] PATH_INFO: `/v3/projects/8a73b10aa92d452a890c7d906b8303c3/users/c1623f961a8b4002a26c50a14cdb85da/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:28.086330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7eedf1-371e-473b-9ec2-9b9afb4c5d1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a73b10aa92d452a890c7d906b8303c3, user_id=c1623f961a8b4002a26c50a14cdb85da, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:28.098483 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7eedf1-371e-473b-9ec2-9b9afb4c5d1e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:28.123326 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39c1ba-2ecf-48c9-96d5-b224839f54d8 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:28.130355 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 888/1760] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:28 2022] PUT /identity/v3/projects/8a73b10aa92d452a890c7d906b8303c3/users/c1623f961a8b4002a26c50a14cdb85da/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:28.131393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7eedf1-371e-473b-9ec2-9b9afb4c5d1e admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:28.265190 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:28.292838 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7eedf1-371e-473b-9ec2-9b9afb4c5d1e admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:28.300363 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff70f1b-f017-45c4-af68-4eedce478b92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:28.301119 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff70f1b-f017-45c4-af68-4eedce478b92 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:28.301411 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff70f1b-f017-45c4-af68-4eedce478b92 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:28.301671 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff70f1b-f017-45c4-af68-4eedce478b92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:28.306723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48511e33-8925-4f4f-a66a-defc5428cbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:28.307415 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48511e33-8925-4f4f-a66a-defc5428cbfb admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:28.307773 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48511e33-8925-4f4f-a66a-defc5428cbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:28.307962 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48511e33-8925-4f4f-a66a-defc5428cbfb admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:28.308704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48511e33-8925-4f4f-a66a-defc5428cbfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:28.314262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48511e33-8925-4f4f-a66a-defc5428cbfb admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:28.335351 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff70f1b-f017-45c4-af68-4eedce478b92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:28.337905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 889/1761] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:28.338161 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff70f1b-f017-45c4-af68-4eedce478b92 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:28.342080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3565ad3-5ecd-4c70-ab56-48e00f699999 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:28.342397 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3565ad3-5ecd-4c70-ab56-48e00f699999 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:28.342658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3565ad3-5ecd-4c70-ab56-48e00f699999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:28.362076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 873/1762] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:28.366186 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff70f1b-f017-45c4-af68-4eedce478b92 service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:28.379556 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff70f1b-f017-45c4-af68-4eedce478b92 service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:28.390290 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31f600-51ae-447e-801a-adf4545498c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:28.391243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31f600-51ae-447e-801a-adf4545498c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:28.391524 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31f600-51ae-447e-801a-adf4545498c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:28.404817 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31f600-51ae-447e-801a-adf4545498c0 admin admin] PATH_INFO: `/v3/projects/fae42fd587ed4ee99b8d0c24a2ff20dd/users/070cde285fe04bc7bd5768fb803a0026/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:28.408126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-f3565ad3-5ecd-4c70-ab56-48e00f699999 None None] MFA Rules not processed for user `c1623f961a8b4002a26c50a14cdb85da`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:28.410161 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f3565ad3-5ecd-4c70-ab56-48e00f699999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:28.435633 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31f600-51ae-447e-801a-adf4545498c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fae42fd587ed4ee99b8d0c24a2ff20dd, user_id=070cde285fe04bc7bd5768fb803a0026, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:28.443697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31f600-51ae-447e-801a-adf4545498c0 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:28.478479 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 874/1763] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:28 2022] PUT /identity/v3/projects/fae42fd587ed4ee99b8d0c24a2ff20dd/users/070cde285fe04bc7bd5768fb803a0026/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:28.489167 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30a257-7e5b-4764-ab67-b2dd9ce7e0ed None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:28.489499 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30a257-7e5b-4764-ab67-b2dd9ce7e0ed None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:28.489725 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30a257-7e5b-4764-ab67-b2dd9ce7e0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:28.547657 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:28.561296 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-ce30a257-7e5b-4764-ab67-b2dd9ce7e0ed None None] MFA Rules not processed for user `070cde285fe04bc7bd5768fb803a0026`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:28.562760 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-ce30a257-7e5b-4764-ab67-b2dd9ce7e0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:28.615992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 890/1764] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:28 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:28.617101 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3565ad3-5ecd-4c70-ab56-48e00f699999 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:28.640812 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f3565ad3-5ecd-4c70-ab56-48e00f699999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:28.680626 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3565ad3-5ecd-4c70-ab56-48e00f699999 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:28.768917 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 875/1765] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:28 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:56:28.770105 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30a257-7e5b-4764-ab67-b2dd9ce7e0ed None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:28.774993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-ce30a257-7e5b-4764-ab67-b2dd9ce7e0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:28.912573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729c856-9d60-42cb-a06e-57c9a2254fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:28.913626 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729c856-9d60-42cb-a06e-57c9a2254fd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:28.913967 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729c856-9d60-42cb-a06e-57c9a2254fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:28.914285 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729c856-9d60-42cb-a06e-57c9a2254fd7 admin admin] PATH_INFO: `/v3/projects/b634466941414d0f8b36d0c3f1af329d` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:28.920236 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729c856-9d60-42cb-a06e-57c9a2254fd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b634466941414d0f8b36d0c3f1af329d)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:28.926756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729c856-9d60-42cb-a06e-57c9a2254fd7 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:29.003658 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30a257-7e5b-4764-ab67-b2dd9ce7e0ed None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:29.061491 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1729c856-9d60-42cb-a06e-57c9a2254fd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b634466941414d0f8b36d0c3f1af329d', 'request_id': 'req-1729c856-9d60-42cb-a06e-57c9a2254fd7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:29.087682 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1729c856-9d60-42cb-a06e-57c9a2254fd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b634466941414d0f8b36d0c3f1af329d', 'request_id': 'req-1729c856-9d60-42cb-a06e-57c9a2254fd7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:29.105733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1729c856-9d60-42cb-a06e-57c9a2254fd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b634466941414d0f8b36d0c3f1af329d', 'request_id': 'req-1729c856-9d60-42cb-a06e-57c9a2254fd7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:29.153929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1729c856-9d60-42cb-a06e-57c9a2254fd7 admin admin] The token cache is being invalidate because project b634466941414d0f8b36d0c3f1af329d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:29.154147 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1729c856-9d60-42cb-a06e-57c9a2254fd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:29.179026 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1729c856-9d60-42cb-a06e-57c9a2254fd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:29.187008 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 891/1766] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:56:28 2022] DELETE /identity/v3/projects/b634466941414d0f8b36d0c3f1af329d => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:29.189340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729c856-9d60-42cb-a06e-57c9a2254fd7 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:29.196690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1729c856-9d60-42cb-a06e-57c9a2254fd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:29.229696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea51dbe-4f47-4ca5-af66-cea807413391 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:29.233368 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea51dbe-4f47-4ca5-af66-cea807413391 service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:29.233899 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea51dbe-4f47-4ca5-af66-cea807413391 service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:29.236075 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea51dbe-4f47-4ca5-af66-cea807413391 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:29.240893 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-bea51dbe-4f47-4ca5-af66-cea807413391 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:29.327945 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:29.521398 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea51dbe-4f47-4ca5-af66-cea807413391 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:29.526092 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea51dbe-4f47-4ca5-af66-cea807413391 service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:29.564117 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729c856-9d60-42cb-a06e-57c9a2254fd7 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:29.600890 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b035218-9be6-476d-8222-b37739e8ecf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:29.607624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b035218-9be6-476d-8222-b37739e8ecf5 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:29.608404 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b035218-9be6-476d-8222-b37739e8ecf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:29.608994 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b035218-9be6-476d-8222-b37739e8ecf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:29.610668 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 876/1767] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:28 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 841 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:29.611062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b035218-9be6-476d-8222-b37739e8ecf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:29.613507 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea51dbe-4f47-4ca5-af66-cea807413391 service cinder] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:29.616669 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b035218-9be6-476d-8222-b37739e8ecf5 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:29.620037 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-bea51dbe-4f47-4ca5-af66-cea807413391 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:29.698671 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea51dbe-4f47-4ca5-af66-cea807413391 service cinder] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:29.710633 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:29.724983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08538a-e92f-4e87-8ac0-c6fea44cc0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:29.742882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08538a-e92f-4e87-8ac0-c6fea44cc0e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:29.745099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08538a-e92f-4e87-8ac0-c6fea44cc0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:29.747339 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08538a-e92f-4e87-8ac0-c6fea44cc0e3 admin admin] PATH_INFO: `/v3/users/e72ccf33ec83465398b81653fa81b57c` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:29.757881 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08538a-e92f-4e87-8ac0-c6fea44cc0e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e72ccf33ec83465398b81653fa81b57c)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:29.780023 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 892/1768] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:56:29 2022] POST /identity/v3/projects => generated 363 bytes in 582 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:29.781274 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08538a-e92f-4e87-8ac0-c6fea44cc0e3 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:29.781871 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b035218-9be6-476d-8222-b37739e8ecf5 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:29.967127 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-0e08538a-e92f-4e87-8ac0-c6fea44cc0e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e72ccf33ec83465398b81653fa81b57c'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:30.000170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-0e08538a-e92f-4e87-8ac0-c6fea44cc0e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e72ccf33ec83465398b81653fa81b57c'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:30.011960 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-0e08538a-e92f-4e87-8ac0-c6fea44cc0e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e72ccf33ec83465398b81653fa81b57c'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:30.013261 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-0e08538a-e92f-4e87-8ac0-c6fea44cc0e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e72ccf33ec83465398b81653fa81b57c'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:30.014566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-0e08538a-e92f-4e87-8ac0-c6fea44cc0e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e72ccf33ec83465398b81653fa81b57c'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:30.057997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 877/1769] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:56:29 2022] DELETE /identity/v3/users/e72ccf33ec83465398b81653fa81b57c => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:30.058478 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08538a-e92f-4e87-8ac0-c6fea44cc0e3 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:30.059618 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b035218-9be6-476d-8222-b37739e8ecf5 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:30.062210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-0e08538a-e92f-4e87-8ac0-c6fea44cc0e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:30.238029 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08538a-e92f-4e87-8ac0-c6fea44cc0e3 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:30.238029 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec292d4-0507-414a-8969-cae1db4267a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:30.238029 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec292d4-0507-414a-8969-cae1db4267a6 service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:30.238029 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec292d4-0507-414a-8969-cae1db4267a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:30.238491 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec292d4-0507-414a-8969-cae1db4267a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:30.239929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba909a3e-c629-4f36-bdb5-cc2817aa82c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:30.240913 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba909a3e-c629-4f36-bdb5-cc2817aa82c0 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:30.241726 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba909a3e-c629-4f36-bdb5-cc2817aa82c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:30.242058 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba909a3e-c629-4f36-bdb5-cc2817aa82c0 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:30.243691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-cec292d4-0507-414a-8969-cae1db4267a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:30.244781 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba909a3e-c629-4f36-bdb5-cc2817aa82c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:30.250711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba909a3e-c629-4f36-bdb5-cc2817aa82c0 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:30.374562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 893/1770] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:56:29 2022] POST /identity/v3/users => generated 288 bytes in 594 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:56:30.405978 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba909a3e-c629-4f36-bdb5-cc2817aa82c0 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:30.417672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ba909a3e-c629-4f36-bdb5-cc2817aa82c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:30.432688 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:30.462281 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba909a3e-c629-4f36-bdb5-cc2817aa82c0 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:30.475400 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949b03d-1c20-47f6-8345-9a88b39119ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:30.477908 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949b03d-1c20-47f6-8345-9a88b39119ba admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:30.478308 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949b03d-1c20-47f6-8345-9a88b39119ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:30.478604 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949b03d-1c20-47f6-8345-9a88b39119ba admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:30.481434 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949b03d-1c20-47f6-8345-9a88b39119ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:30.483214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949b03d-1c20-47f6-8345-9a88b39119ba admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:30.491657 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec292d4-0507-414a-8969-cae1db4267a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:30.494440 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec292d4-0507-414a-8969-cae1db4267a6 service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:30.520465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 894/1771] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:30.524199 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949b03d-1c20-47f6-8345-9a88b39119ba admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:30.527892 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 878/1772] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:30 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:30.539023 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949b03d-1c20-47f6-8345-9a88b39119ba admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:30.560983 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747b2d1-ec32-46d9-9e0a-20eaf6b5229d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:30.561906 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747b2d1-ec32-46d9-9e0a-20eaf6b5229d admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:30.567076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747b2d1-ec32-46d9-9e0a-20eaf6b5229d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:30.567418 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747b2d1-ec32-46d9-9e0a-20eaf6b5229d admin admin] PATH_INFO: `/v3/projects/59023bd2b85c40998f4e83c41b5bc462/users/116802c8a90d4990ba762b3d39d29cb2/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:30.622696 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747b2d1-ec32-46d9-9e0a-20eaf6b5229d admin admin] RBAC: Authorizing `identity:create_grant(project_id=59023bd2b85c40998f4e83c41b5bc462, user_id=116802c8a90d4990ba762b3d39d29cb2, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:30.629317 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747b2d1-ec32-46d9-9e0a-20eaf6b5229d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:30.647214 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2eecd2-3757-4916-850a-8e465bf454de None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:30.647851 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2eecd2-3757-4916-850a-8e465bf454de None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:30.648306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2eecd2-3757-4916-850a-8e465bf454de None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:30.650061 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 879/1773] 10.222.0.78 () {54 vars in 921 bytes} [Wed May 25 14:56:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:56:30.696344 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 895/1774] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:30 2022] PUT /identity/v3/projects/59023bd2b85c40998f4e83c41b5bc462/users/116802c8a90d4990ba762b3d39d29cb2/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:30.700619 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2eecd2-3757-4916-850a-8e465bf454de None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:30.769303 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87ce07-7105-405c-afec-a89cbec13011 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:30.770167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87ce07-7105-405c-afec-a89cbec13011 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:30.770568 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87ce07-7105-405c-afec-a89cbec13011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:30.869161 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-ee87ce07-7105-405c-afec-a89cbec13011 None None] MFA Rules not processed for user `e34e1d7ffbac49b0bde45829af0c6925`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:30.879055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ee87ce07-7105-405c-afec-a89cbec13011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:31.035527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2eecd2-3757-4916-850a-8e465bf454de None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:31.052619 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e82b0e-1650-446b-a360-aabc961d9f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:31.053850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e82b0e-1650-446b-a360-aabc961d9f3b admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:31.054257 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e82b0e-1650-446b-a360-aabc961d9f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:31.054593 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e82b0e-1650-446b-a360-aabc961d9f3b admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:31.055600 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e82b0e-1650-446b-a360-aabc961d9f3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:31.058524 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e82b0e-1650-446b-a360-aabc961d9f3b admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:31.093785 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 880/1775] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:31.094872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e82b0e-1650-446b-a360-aabc961d9f3b admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:31.113552 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:31.121537 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e82b0e-1650-446b-a360-aabc961d9f3b admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:31.138994 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba35319b-4083-468f-afe9-855cefd24d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:31.139917 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba35319b-4083-468f-afe9-855cefd24d6b admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:31.140198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba35319b-4083-468f-afe9-855cefd24d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:31.140471 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba35319b-4083-468f-afe9-855cefd24d6b admin admin] PATH_INFO: `/v3/projects/59023bd2b85c40998f4e83c41b5bc462/users/116802c8a90d4990ba762b3d39d29cb2/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:31.144239 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba35319b-4083-468f-afe9-855cefd24d6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=59023bd2b85c40998f4e83c41b5bc462, user_id=116802c8a90d4990ba762b3d39d29cb2, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:31.147656 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba35319b-4083-468f-afe9-855cefd24d6b admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:31.169621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 896/1776] 10.222.0.78 () {58 vars in 1018 bytes} [Wed May 25 14:56:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:56:31.197028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 881/1777] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:31 2022] PUT /identity/v3/projects/59023bd2b85c40998f4e83c41b5bc462/users/116802c8a90d4990ba762b3d39d29cb2/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:31.203765 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7076a6-8580-4d0a-9a50-832b997ee6c5 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:31.212278 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7076a6-8580-4d0a-9a50-832b997ee6c5 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:31.212672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7076a6-8580-4d0a-9a50-832b997ee6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:31.329238 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-0f7076a6-8580-4d0a-9a50-832b997ee6c5 None None] MFA Rules not processed for user `116802c8a90d4990ba762b3d39d29cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:31.337620 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-0f7076a6-8580-4d0a-9a50-832b997ee6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:31.695424 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba35319b-4083-468f-afe9-855cefd24d6b admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:31.697225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-ba35319b-4083-468f-afe9-855cefd24d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:31.732101 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 897/1778] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:31 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 529 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:31.748302 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7076a6-8580-4d0a-9a50-832b997ee6c5 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:31.950790 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7076a6-8580-4d0a-9a50-832b997ee6c5 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:31.955372 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba35319b-4083-468f-afe9-855cefd24d6b admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:31.970097 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b103be-e19d-4999-9637-5aefa751b00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:31.971069 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b103be-e19d-4999-9637-5aefa751b00e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:31.971203 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b103be-e19d-4999-9637-5aefa751b00e admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:31.971503 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b103be-e19d-4999-9637-5aefa751b00e admin admin] PATH_INFO: `/v3/projects/5fbc0fd1768644eab3fcabc81eed84f6` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:31.973071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b103be-e19d-4999-9637-5aefa751b00e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fbc0fd1768644eab3fcabc81eed84f6)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:31.975064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b103be-e19d-4999-9637-5aefa751b00e admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:31.985756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ceafd7-c6c1-4496-85e6-70ec8f39fc2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:31.986524 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ceafd7-c6c1-4496-85e6-70ec8f39fc2c service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:31.987044 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ceafd7-c6c1-4496-85e6-70ec8f39fc2c service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:31.997603 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ceafd7-c6c1-4496-85e6-70ec8f39fc2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:32.079304 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ceafd7-c6c1-4496-85e6-70ec8f39fc2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:32.086910 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ceafd7-c6c1-4496-85e6-70ec8f39fc2c service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:32.107838 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d1b103be-e19d-4999-9637-5aefa751b00e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fbc0fd1768644eab3fcabc81eed84f6', 'request_id': 'req-d1b103be-e19d-4999-9637-5aefa751b00e'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:32.122677 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d1b103be-e19d-4999-9637-5aefa751b00e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fbc0fd1768644eab3fcabc81eed84f6', 'request_id': 'req-d1b103be-e19d-4999-9637-5aefa751b00e'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:32.126477 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 898/1779] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:31 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:32.135909 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d1b103be-e19d-4999-9637-5aefa751b00e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fbc0fd1768644eab3fcabc81eed84f6', 'request_id': 'req-d1b103be-e19d-4999-9637-5aefa751b00e'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:32.138747 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:32.173026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d1b103be-e19d-4999-9637-5aefa751b00e admin admin] The token cache is being invalidate because project 5fbc0fd1768644eab3fcabc81eed84f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:32.173026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d1b103be-e19d-4999-9637-5aefa751b00e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:32.183746 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d1b103be-e19d-4999-9637-5aefa751b00e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:32.190337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 882/1780] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:56:31 2022] DELETE /identity/v3/projects/5fbc0fd1768644eab3fcabc81eed84f6 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:56:32.194323 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ceafd7-c6c1-4496-85e6-70ec8f39fc2c service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:32.213876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b103be-e19d-4999-9637-5aefa751b00e admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:32.216295 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d1b103be-e19d-4999-9637-5aefa751b00e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:32.219699 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ceafd7-c6c1-4496-85e6-70ec8f39fc2c service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:32.341143 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e45fd-5b0d-4560-b3d0-41444fed2019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:32.342109 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e45fd-5b0d-4560-b3d0-41444fed2019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:32.342398 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e45fd-5b0d-4560-b3d0-41444fed2019 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:32.342676 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e45fd-5b0d-4560-b3d0-41444fed2019 admin admin] PATH_INFO: `/v3/users/cdfeeeabebd94fd78ecc803679e61dbf` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:32.343920 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e45fd-5b0d-4560-b3d0-41444fed2019 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdfeeeabebd94fd78ecc803679e61dbf)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:32.346245 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e45fd-5b0d-4560-b3d0-41444fed2019 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:32.409236 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:32.433313 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b103be-e19d-4999-9637-5aefa751b00e admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:32.441939 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001520a4-c0d3-4b79-9895-69d90bc2e0e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:32.443259 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001520a4-c0d3-4b79-9895-69d90bc2e0e7 service nova] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:32.443431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001520a4-c0d3-4b79-9895-69d90bc2e0e7 service nova] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:32.443753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001520a4-c0d3-4b79-9895-69d90bc2e0e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:32.459174 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-616e45fd-5b0d-4560-b3d0-41444fed2019 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdfeeeabebd94fd78ecc803679e61dbf'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:32.460393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-001520a4-c0d3-4b79-9895-69d90bc2e0e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:32.492509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001520a4-c0d3-4b79-9895-69d90bc2e0e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:32.492509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001520a4-c0d3-4b79-9895-69d90bc2e0e7 service nova] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:32.500560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-616e45fd-5b0d-4560-b3d0-41444fed2019 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdfeeeabebd94fd78ecc803679e61dbf'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:32.515613 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-616e45fd-5b0d-4560-b3d0-41444fed2019 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdfeeeabebd94fd78ecc803679e61dbf'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:32.516966 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-616e45fd-5b0d-4560-b3d0-41444fed2019 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdfeeeabebd94fd78ecc803679e61dbf'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:32.517766 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-616e45fd-5b0d-4560-b3d0-41444fed2019 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdfeeeabebd94fd78ecc803679e61dbf'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:32.544543 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 883/1781] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:32.545541 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001520a4-c0d3-4b79-9895-69d90bc2e0e7 service nova] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:32.555063 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-001520a4-c0d3-4b79-9895-69d90bc2e0e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:32.575627 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 899/1782] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:56:32 2022] DELETE /identity/v3/users/cdfeeeabebd94fd78ecc803679e61dbf => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:32.575627 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e45fd-5b0d-4560-b3d0-41444fed2019 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:32.578330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-616e45fd-5b0d-4560-b3d0-41444fed2019 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:32.794454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001520a4-c0d3-4b79-9895-69d90bc2e0e7 service nova] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:32.813100 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e45fd-5b0d-4560-b3d0-41444fed2019 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:32.830872 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1bcf10-e79a-4124-86bd-26388b70d59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:32.831943 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bcf10-e79a-4124-86bd-26388b70d59b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:32.832128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bcf10-e79a-4124-86bd-26388b70d59b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:32.832408 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bcf10-e79a-4124-86bd-26388b70d59b admin admin] PATH_INFO: `/v3/projects/806c800509b340a0b4768da6251fb7a0` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:32.838634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1bcf10-e79a-4124-86bd-26388b70d59b admin admin] RBAC: Authorizing `identity:delete_project(project_id=806c800509b340a0b4768da6251fb7a0)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:32.841618 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1bcf10-e79a-4124-86bd-26388b70d59b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:32.944279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fdc665-b11c-428e-a589-e2b9f64c9d4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:32.945256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fdc665-b11c-428e-a589-e2b9f64c9d4b service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:32.945546 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fdc665-b11c-428e-a589-e2b9f64c9d4b service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:32.945827 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fdc665-b11c-428e-a589-e2b9f64c9d4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:32.947956 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-38fdc665-b11c-428e-a589-e2b9f64c9d4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:33.031576 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-de1bcf10-e79a-4124-86bd-26388b70d59b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '806c800509b340a0b4768da6251fb7a0', 'request_id': 'req-de1bcf10-e79a-4124-86bd-26388b70d59b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:33.050731 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-de1bcf10-e79a-4124-86bd-26388b70d59b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '806c800509b340a0b4768da6251fb7a0', 'request_id': 'req-de1bcf10-e79a-4124-86bd-26388b70d59b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:33.063398 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:33.067720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-de1bcf10-e79a-4124-86bd-26388b70d59b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '806c800509b340a0b4768da6251fb7a0', 'request_id': 'req-de1bcf10-e79a-4124-86bd-26388b70d59b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:33.125880 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-de1bcf10-e79a-4124-86bd-26388b70d59b admin admin] The token cache is being invalidate because project 806c800509b340a0b4768da6251fb7a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:33.128342 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-de1bcf10-e79a-4124-86bd-26388b70d59b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:33.133449 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fdc665-b11c-428e-a589-e2b9f64c9d4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:33.134499 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-de1bcf10-e79a-4124-86bd-26388b70d59b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:33.138678 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 900/1783] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:56:32 2022] DELETE /identity/v3/projects/806c800509b340a0b4768da6251fb7a0 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:33.138958 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fdc665-b11c-428e-a589-e2b9f64c9d4b service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:33.143602 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cb42d-606e-4b48-918e-842b2f615d78 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:33.143890 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cb42d-606e-4b48-918e-842b2f615d78 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:33.144268 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cb42d-606e-4b48-918e-842b2f615d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:33.163869 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 884/1784] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:32 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:33.178607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-5c8cb42d-606e-4b48-918e-842b2f615d78 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:33.180161 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5c8cb42d-606e-4b48-918e-842b2f615d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:33.236763 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:33.395268 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 901/1785] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:33.397107 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ac070-5881-4cc7-b454-a120391f8e89 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:33.397407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ac070-5881-4cc7-b454-a120391f8e89 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:33.397689 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ac070-5881-4cc7-b454-a120391f8e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:33.459172 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-892ac070-5881-4cc7-b454-a120391f8e89 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:33.462302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-892ac070-5881-4cc7-b454-a120391f8e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:33.488847 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 885/1786] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:56:33.490905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8cb42d-606e-4b48-918e-842b2f615d78 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:33.513800 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8cb42d-606e-4b48-918e-842b2f615d78 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:33.519813 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76adb5f5-2cf0-452e-a062-85158a7101a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:33.520527 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adb5f5-2cf0-452e-a062-85158a7101a9 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:33.520830 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adb5f5-2cf0-452e-a062-85158a7101a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:33.521111 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adb5f5-2cf0-452e-a062-85158a7101a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:33.522092 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76adb5f5-2cf0-452e-a062-85158a7101a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:56:33.522362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76adb5f5-2cf0-452e-a062-85158a7101a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:33.524369 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76adb5f5-2cf0-452e-a062-85158a7101a9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:33.553180 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 902/1787] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:56:33.558510 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ac070-5881-4cc7-b454-a120391f8e89 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:33.570183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ac070-5881-4cc7-b454-a120391f8e89 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:33.577826 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574ae6f-8f96-4995-b310-6b0e2c298048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:33.578471 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574ae6f-8f96-4995-b310-6b0e2c298048 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:33.578754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574ae6f-8f96-4995-b310-6b0e2c298048 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:33.579033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574ae6f-8f96-4995-b310-6b0e2c298048 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:33.579823 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574ae6f-8f96-4995-b310-6b0e2c298048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:33.581971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574ae6f-8f96-4995-b310-6b0e2c298048 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:33.640938 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76adb5f5-2cf0-452e-a062-85158a7101a9 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:33.641257 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 886/1788] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:56:33 2022] POST /identity/v3/projects => generated 373 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:33.657080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76adb5f5-2cf0-452e-a062-85158a7101a9 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:33.756147 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa713f83-2b88-4abf-9801-b4dca95dc287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:33.757205 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa713f83-2b88-4abf-9801-b4dca95dc287 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:33.757800 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa713f83-2b88-4abf-9801-b4dca95dc287 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:33.758232 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa713f83-2b88-4abf-9801-b4dca95dc287 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:33.759356 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa713f83-2b88-4abf-9801-b4dca95dc287 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:33.761794 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa713f83-2b88-4abf-9801-b4dca95dc287 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:33.795106 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574ae6f-8f96-4995-b310-6b0e2c298048 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:33.796699 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 903/1789] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:56:33 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:56:33.799261 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-4574ae6f-8f96-4995-b310-6b0e2c298048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:33.808315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa713f83-2b88-4abf-9801-b4dca95dc287 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:33.817405 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574ae6f-8f96-4995-b310-6b0e2c298048 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:33.823045 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd59f79-173f-4c83-9c76-099f501c73d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:33.824306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd59f79-173f-4c83-9c76-099f501c73d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:33.825613 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd59f79-173f-4c83-9c76-099f501c73d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:33.826230 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd59f79-173f-4c83-9c76-099f501c73d1 admin admin] PATH_INFO: `/v3/projects/0283df17437544bbb5b3452480958c06` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:33.828018 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa713f83-2b88-4abf-9801-b4dca95dc287 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:33.830732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd59f79-173f-4c83-9c76-099f501c73d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0283df17437544bbb5b3452480958c06)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:33.837553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2635e-f4fb-439d-a721-6c4275d3c6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:33.838404 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2635e-f4fb-439d-a721-6c4275d3c6f5 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:33.838690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2635e-f4fb-439d-a721-6c4275d3c6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:33.838967 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2635e-f4fb-439d-a721-6c4275d3c6f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:33.841804 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2635e-f4fb-439d-a721-6c4275d3c6f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:33.842158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd59f79-173f-4c83-9c76-099f501c73d1 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:33.844126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2635e-f4fb-439d-a721-6c4275d3c6f5 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:33.859520 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 904/1790] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:33.866667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2635e-f4fb-439d-a721-6c4275d3c6f5 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:33.897489 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2635e-f4fb-439d-a721-6c4275d3c6f5 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:33.913447 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb30f2-83cf-4ea4-b469-1ce542365d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:33.914309 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb30f2-83cf-4ea4-b469-1ce542365d09 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:33.914649 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb30f2-83cf-4ea4-b469-1ce542365d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:33.915001 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb30f2-83cf-4ea4-b469-1ce542365d09 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:33.920023 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb30f2-83cf-4ea4-b469-1ce542365d09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:33.924541 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb30f2-83cf-4ea4-b469-1ce542365d09 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:33.972812 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 905/1791] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:33.975649 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb30f2-83cf-4ea4-b469-1ce542365d09 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:33.988725 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3bd59f79-173f-4c83-9c76-099f501c73d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0283df17437544bbb5b3452480958c06', 'request_id': 'req-3bd59f79-173f-4c83-9c76-099f501c73d1'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:34.002951 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3bd59f79-173f-4c83-9c76-099f501c73d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0283df17437544bbb5b3452480958c06', 'request_id': 'req-3bd59f79-173f-4c83-9c76-099f501c73d1'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:34.011034 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb30f2-83cf-4ea4-b469-1ce542365d09 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:34.017590 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3bd59f79-173f-4c83-9c76-099f501c73d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0283df17437544bbb5b3452480958c06', 'request_id': 'req-3bd59f79-173f-4c83-9c76-099f501c73d1'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:34.044134 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3bd59f79-173f-4c83-9c76-099f501c73d1 admin admin] The token cache is being invalidate because project 0283df17437544bbb5b3452480958c06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:34.044558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3bd59f79-173f-4c83-9c76-099f501c73d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:34.050748 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3bd59f79-173f-4c83-9c76-099f501c73d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:34.060273 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 887/1792] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:56:33 2022] DELETE /identity/v3/projects/0283df17437544bbb5b3452480958c06 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:34.060752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcc3c9-bb26-4917-97a1-dd6e2a36b1dc None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:34.061040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcc3c9-bb26-4917-97a1-dd6e2a36b1dc None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:34.061320 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcc3c9-bb26-4917-97a1-dd6e2a36b1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:34.112550 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-06bcc3c9-bb26-4917-97a1-dd6e2a36b1dc None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:34.120705 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-06bcc3c9-bb26-4917-97a1-dd6e2a36b1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:34.129181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1db838-05b9-4cdf-93c2-3f435aea83bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:34.130568 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1db838-05b9-4cdf-93c2-3f435aea83bc admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:34.131108 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1db838-05b9-4cdf-93c2-3f435aea83bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:34.131544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1db838-05b9-4cdf-93c2-3f435aea83bc admin admin] PATH_INFO: `/v3/projects/66d1682b757348df9fb1240bf9a7edd9/users/b140f43a113849869e5f7bf9b55bc8fa/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:34.152837 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1db838-05b9-4cdf-93c2-3f435aea83bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=66d1682b757348df9fb1240bf9a7edd9, user_id=b140f43a113849869e5f7bf9b55bc8fa, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:34.160254 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 888/1793] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:34.164601 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275b1bd-51d6-427f-a03a-c5ed55b4d38d None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:34.165120 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1db838-05b9-4cdf-93c2-3f435aea83bc admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:34.165674 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275b1bd-51d6-427f-a03a-c5ed55b4d38d None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:34.166454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275b1bd-51d6-427f-a03a-c5ed55b4d38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:34.199007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 906/1794] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:33 2022] PUT /identity/v3/projects/66d1682b757348df9fb1240bf9a7edd9/users/b140f43a113849869e5f7bf9b55bc8fa/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:34.210779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98874c51-87c9-458c-a7d5-abade274d33f None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:34.211303 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98874c51-87c9-458c-a7d5-abade274d33f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:34.211732 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98874c51-87c9-458c-a7d5-abade274d33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:34.242740 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-6275b1bd-51d6-427f-a03a-c5ed55b4d38d None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:34.244991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-6275b1bd-51d6-427f-a03a-c5ed55b4d38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:34.249921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:34.312554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-98874c51-87c9-458c-a7d5-abade274d33f None None] MFA Rules not processed for user `b140f43a113849869e5f7bf9b55bc8fa`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:34.314052 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-98874c51-87c9-458c-a7d5-abade274d33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:34.456119 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 889/1795] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:34.460365 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275b1bd-51d6-427f-a03a-c5ed55b4d38d None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:34.480156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 907/1796] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:34.481932 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275b1bd-51d6-427f-a03a-c5ed55b4d38d None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:34.485227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98874c51-87c9-458c-a7d5-abade274d33f None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:34.487728 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-98874c51-87c9-458c-a7d5-abade274d33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:34.491244 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ea20d-7226-4976-92a2-4f7ff03bf781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:34.493055 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ea20d-7226-4976-92a2-4f7ff03bf781 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:34.493783 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ea20d-7226-4976-92a2-4f7ff03bf781 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:34.494364 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ea20d-7226-4976-92a2-4f7ff03bf781 admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:34.495505 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7ea20d-7226-4976-92a2-4f7ff03bf781 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:56:34.496186 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7ea20d-7226-4976-92a2-4f7ff03bf781 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:34.501788 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7ea20d-7226-4976-92a2-4f7ff03bf781 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:34.521474 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98874c51-87c9-458c-a7d5-abade274d33f None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:34.542341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cab133-ccd4-4ba7-8f9e-ea897aa7b460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:34.543042 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cab133-ccd4-4ba7-8f9e-ea897aa7b460 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:34.544076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 890/1797] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:56:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:56:34.544415 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cab133-ccd4-4ba7-8f9e-ea897aa7b460 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:34.544705 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cab133-ccd4-4ba7-8f9e-ea897aa7b460 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:34.545701 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cab133-ccd4-4ba7-8f9e-ea897aa7b460 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:34.547678 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ea20d-7226-4976-92a2-4f7ff03bf781 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:34.555433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cab133-ccd4-4ba7-8f9e-ea897aa7b460 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:34.635616 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ea20d-7226-4976-92a2-4f7ff03bf781 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:34.665032 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f3343-caf3-4338-9c3e-79668c1b1011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:34.667017 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f3343-caf3-4338-9c3e-79668c1b1011 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:34.667017 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f3343-caf3-4338-9c3e-79668c1b1011 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:34.667715 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f3343-caf3-4338-9c3e-79668c1b1011 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:34.669635 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f3343-caf3-4338-9c3e-79668c1b1011 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:34.673075 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f3343-caf3-4338-9c3e-79668c1b1011 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:34.694875 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 908/1798] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:56:34 2022] POST /identity/v3/projects => generated 373 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:34.696060 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cab133-ccd4-4ba7-8f9e-ea897aa7b460 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:34.702680 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-21cab133-ccd4-4ba7-8f9e-ea897aa7b460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:34.736413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:34.760633 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 891/1799] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:56:34 2022] POST /identity/v3/projects => generated 369 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:34.761652 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f3343-caf3-4338-9c3e-79668c1b1011 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:34.900619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cab133-ccd4-4ba7-8f9e-ea897aa7b460 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:34.942709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f3343-caf3-4338-9c3e-79668c1b1011 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:34.964310 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b36a9-ab66-4880-8474-594ef525fa3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:34.965435 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b36a9-ab66-4880-8474-594ef525fa3c admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:34.966569 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b36a9-ab66-4880-8474-594ef525fa3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:34.968041 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b36a9-ab66-4880-8474-594ef525fa3c admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:34.971358 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b36a9-ab66-4880-8474-594ef525fa3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:34.977583 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b36a9-ab66-4880-8474-594ef525fa3c admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:35.032643 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 892/1800] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:56:34 2022] POST /identity/v3/users => generated 293 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:35.037429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b36a9-ab66-4880-8474-594ef525fa3c admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:35.091113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998f98f-f9f9-4ffd-b626-8c96cd525a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:35.091997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b36a9-ab66-4880-8474-594ef525fa3c admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:35.095230 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998f98f-f9f9-4ffd-b626-8c96cd525a02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:35.095604 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998f98f-f9f9-4ffd-b626-8c96cd525a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:35.096065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998f98f-f9f9-4ffd-b626-8c96cd525a02 admin admin] PATH_INFO: `/v3/users/b54594f051334e0282f146c626064717` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:35.099022 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5998f98f-f9f9-4ffd-b626-8c96cd525a02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b54594f051334e0282f146c626064717)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:35.102662 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5998f98f-f9f9-4ffd-b626-8c96cd525a02 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:35.118086 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfda7d-7c49-4986-b2de-04544dc18f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:35.123815 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfda7d-7c49-4986-b2de-04544dc18f18 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:35.126799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfda7d-7c49-4986-b2de-04544dc18f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:35.126938 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfda7d-7c49-4986-b2de-04544dc18f18 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:35.131885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfda7d-7c49-4986-b2de-04544dc18f18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:35.140268 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfda7d-7c49-4986-b2de-04544dc18f18 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:35.208574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:35.213669 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 893/1801] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:56:35 2022] POST /identity/v3/users => generated 285 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:35.214752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfda7d-7c49-4986-b2de-04544dc18f18 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:35.233526 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfda7d-7c49-4986-b2de-04544dc18f18 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:35.240433 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689c3ed-fbe4-4686-bb54-4c417a05f314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:35.241244 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689c3ed-fbe4-4686-bb54-4c417a05f314 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:35.241675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689c3ed-fbe4-4686-bb54-4c417a05f314 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:35.242535 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689c3ed-fbe4-4686-bb54-4c417a05f314 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:35.244007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e689c3ed-fbe4-4686-bb54-4c417a05f314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:35.248750 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e689c3ed-fbe4-4686-bb54-4c417a05f314 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:35.264435 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 894/1802] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:35.268634 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689c3ed-fbe4-4686-bb54-4c417a05f314 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:35.280966 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5998f98f-f9f9-4ffd-b626-8c96cd525a02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b54594f051334e0282f146c626064717'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:35.288712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689c3ed-fbe4-4686-bb54-4c417a05f314 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:35.300085 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53849ae-61d1-415b-976b-eb33eced5264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:35.300906 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53849ae-61d1-415b-976b-eb33eced5264 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:35.301273 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53849ae-61d1-415b-976b-eb33eced5264 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:35.301642 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53849ae-61d1-415b-976b-eb33eced5264 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:35.303177 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53849ae-61d1-415b-976b-eb33eced5264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:35.305781 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53849ae-61d1-415b-976b-eb33eced5264 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:35.311694 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5998f98f-f9f9-4ffd-b626-8c96cd525a02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b54594f051334e0282f146c626064717'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:35.323571 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 895/1803] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:35.324510 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53849ae-61d1-415b-976b-eb33eced5264 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:35.331657 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5998f98f-f9f9-4ffd-b626-8c96cd525a02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b54594f051334e0282f146c626064717'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:35.332785 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5998f98f-f9f9-4ffd-b626-8c96cd525a02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b54594f051334e0282f146c626064717'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:35.333426 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5998f98f-f9f9-4ffd-b626-8c96cd525a02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b54594f051334e0282f146c626064717'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:35.353769 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53849ae-61d1-415b-976b-eb33eced5264 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:35.362349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 909/1804] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:56:34 2022] DELETE /identity/v3/users/b54594f051334e0282f146c626064717 => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:35.363430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998f98f-f9f9-4ffd-b626-8c96cd525a02 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:35.363838 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd00786-e81d-4b86-ad3a-bf870d96fa1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:35.365047 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd00786-e81d-4b86-ad3a-bf870d96fa1c admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:35.365488 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd00786-e81d-4b86-ad3a-bf870d96fa1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:35.365854 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd00786-e81d-4b86-ad3a-bf870d96fa1c admin admin] PATH_INFO: `/v3/projects/175e1034a7ae4f7a821046b6b678e55a/users/41216984dda84a9c90c7a3db28a0bac1/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:35.370519 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5998f98f-f9f9-4ffd-b626-8c96cd525a02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:35.395395 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd00786-e81d-4b86-ad3a-bf870d96fa1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=175e1034a7ae4f7a821046b6b678e55a, user_id=41216984dda84a9c90c7a3db28a0bac1, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:35.400142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd00786-e81d-4b86-ad3a-bf870d96fa1c admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:35.433639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 896/1805] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:35 2022] PUT /identity/v3/projects/175e1034a7ae4f7a821046b6b678e55a/users/41216984dda84a9c90c7a3db28a0bac1/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:35.435766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b850f4c-98e3-4460-98c4-e4374bcaaaef None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:35.436127 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b850f4c-98e3-4460-98c4-e4374bcaaaef None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:35.436512 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b850f4c-98e3-4460-98c4-e4374bcaaaef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:35.467363 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-0b850f4c-98e3-4460-98c4-e4374bcaaaef None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:35.468977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-0b850f4c-98e3-4460-98c4-e4374bcaaaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:35.551231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998f98f-f9f9-4ffd-b626-8c96cd525a02 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:35.596544 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 897/1806] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:35.596544 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b850f4c-98e3-4460-98c4-e4374bcaaaef None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:35.598496 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-0b850f4c-98e3-4460-98c4-e4374bcaaaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:35.623298 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b850f4c-98e3-4460-98c4-e4374bcaaaef None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:35.632810 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816a4b0-88dc-4472-8dfe-a12dea72d858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:35.634785 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816a4b0-88dc-4472-8dfe-a12dea72d858 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:35.635458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816a4b0-88dc-4472-8dfe-a12dea72d858 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:35.635955 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816a4b0-88dc-4472-8dfe-a12dea72d858 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:35.637850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816a4b0-88dc-4472-8dfe-a12dea72d858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:35.640738 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816a4b0-88dc-4472-8dfe-a12dea72d858 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:35.671083 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 898/1807] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:35.673463 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816a4b0-88dc-4472-8dfe-a12dea72d858 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:35.678039 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-5816a4b0-88dc-4472-8dfe-a12dea72d858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:35.686784 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2899a-7611-4099-be79-660864235d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:35.688001 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2899a-7611-4099-be79-660864235d1c admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:35.688510 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2899a-7611-4099-be79-660864235d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:35.689024 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2899a-7611-4099-be79-660864235d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:35.691209 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae2899a-7611-4099-be79-660864235d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:35.694611 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae2899a-7611-4099-be79-660864235d1c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:35.722294 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 910/1808] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:35.723313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2899a-7611-4099-be79-660864235d1c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:35.749748 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2899a-7611-4099-be79-660864235d1c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:35.767356 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae8728-c054-40bd-9990-e833941e4c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:35.768548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae8728-c054-40bd-9990-e833941e4c03 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:35.768836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae8728-c054-40bd-9990-e833941e4c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:35.773130 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae8728-c054-40bd-9990-e833941e4c03 admin admin] PATH_INFO: `/v3/projects/175e1034a7ae4f7a821046b6b678e55a/users/41216984dda84a9c90c7a3db28a0bac1/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:35.774831 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae8728-c054-40bd-9990-e833941e4c03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=175e1034a7ae4f7a821046b6b678e55a, user_id=41216984dda84a9c90c7a3db28a0bac1, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:35.779992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae8728-c054-40bd-9990-e833941e4c03 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:35.781190 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:35.814048 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 911/1809] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:35 2022] PUT /identity/v3/projects/175e1034a7ae4f7a821046b6b678e55a/users/41216984dda84a9c90c7a3db28a0bac1/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:35.814048 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae8728-c054-40bd-9990-e833941e4c03 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:35.887719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:35.936143 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae8728-c054-40bd-9990-e833941e4c03 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:35.944589 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5bc0d-a3bd-499d-bd95-89f15155c075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:35.946234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5bc0d-a3bd-499d-bd95-89f15155c075 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:35.946712 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5bc0d-a3bd-499d-bd95-89f15155c075 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:35.947360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5bc0d-a3bd-499d-bd95-89f15155c075 admin admin] PATH_INFO: `/v3/projects/101f6923287148ad91c9c23291237a8b/users/218760f94a944b9c8a7bc436f49d97f9/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:35.973018 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816a4b0-88dc-4472-8dfe-a12dea72d858 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:35.978090 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5bc0d-a3bd-499d-bd95-89f15155c075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=101f6923287148ad91c9c23291237a8b, user_id=218760f94a944b9c8a7bc436f49d97f9, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:35.982592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5bc0d-a3bd-499d-bd95-89f15155c075 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:36.020469 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 912/1810] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:35 2022] PUT /identity/v3/projects/101f6923287148ad91c9c23291237a8b/users/218760f94a944b9c8a7bc436f49d97f9/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:36.022463 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed1d76-6a43-4d56-aede-9ece361b2cc0 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:36.025777 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed1d76-6a43-4d56-aede-9ece361b2cc0 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:36.028402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed1d76-6a43-4d56-aede-9ece361b2cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:36.103807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-05ed1d76-6a43-4d56-aede-9ece361b2cc0 None None] MFA Rules not processed for user `41216984dda84a9c90c7a3db28a0bac1`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:36.105309 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-05ed1d76-6a43-4d56-aede-9ece361b2cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:36.212553 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73e06b-4367-4986-afff-c6f35c306c9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:36.213981 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73e06b-4367-4986-afff-c6f35c306c9c service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:36.214991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73e06b-4367-4986-afff-c6f35c306c9c service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:36.219199 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73e06b-4367-4986-afff-c6f35c306c9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:36.265901 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73e06b-4367-4986-afff-c6f35c306c9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:36.294313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 913/1811] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:36.294313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29776c3-80b3-4640-99a9-d0fda190f23a None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:36.294313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29776c3-80b3-4640-99a9-d0fda190f23a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:36.294313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29776c3-80b3-4640-99a9-d0fda190f23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:36.295423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73e06b-4367-4986-afff-c6f35c306c9c service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:36.332381 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 899/1812] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 660 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:36.353484 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73e06b-4367-4986-afff-c6f35c306c9c service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:36.355640 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-5d73e06b-4367-4986-afff-c6f35c306c9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:36.401328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-c29776c3-80b3-4640-99a9-d0fda190f23a None None] MFA Rules not processed for user `218760f94a944b9c8a7bc436f49d97f9`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:36.402656 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c29776c3-80b3-4640-99a9-d0fda190f23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:36.491776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73e06b-4367-4986-afff-c6f35c306c9c service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:36.562116 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 914/1813] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:36 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:36.564372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29776c3-80b3-4640-99a9-d0fda190f23a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:36.631427 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76ecf1-f820-4336-9802-fcd5face3163 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:36.632001 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76ecf1-f820-4336-9802-fcd5face3163 service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:36.632284 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76ecf1-f820-4336-9802-fcd5face3163 service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:36.632563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76ecf1-f820-4336-9802-fcd5face3163 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:36.649021 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:36.659083 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76ecf1-f820-4336-9802-fcd5face3163 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:36.660351 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76ecf1-f820-4336-9802-fcd5face3163 service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:36.679857 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 900/1814] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:36 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:36.779871 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29776c3-80b3-4640-99a9-d0fda190f23a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:36.810489 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053963e7-9792-47f1-95b7-cc30c5858d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:36.811710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053963e7-9792-47f1-95b7-cc30c5858d49 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:36.812203 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053963e7-9792-47f1-95b7-cc30c5858d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:36.812713 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053963e7-9792-47f1-95b7-cc30c5858d49 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:36.814075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053963e7-9792-47f1-95b7-cc30c5858d49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:36.817593 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053963e7-9792-47f1-95b7-cc30c5858d49 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:36.870460 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76ecf1-f820-4336-9802-fcd5face3163 service cinder] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:36.889565 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76ecf1-f820-4336-9802-fcd5face3163 service cinder] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:36.894545 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4427600-070f-4f43-b3eb-4ded7f8e4442 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:36.895310 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4427600-070f-4f43-b3eb-4ded7f8e4442 service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:36.895652 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4427600-070f-4f43-b3eb-4ded7f8e4442 service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:36.896000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4427600-070f-4f43-b3eb-4ded7f8e4442 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:36.899591 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-c4427600-070f-4f43-b3eb-4ded7f8e4442 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:36.971001 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 915/1815] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:56:36 2022] POST /identity/v3/projects => generated 367 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:36.977070 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053963e7-9792-47f1-95b7-cc30c5858d49 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:36.998465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:37.070797 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4427600-070f-4f43-b3eb-4ded7f8e4442 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:37.074348 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4427600-070f-4f43-b3eb-4ded7f8e4442 service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:37.090249 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 901/1816] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:56:37.092194 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb10701-bc44-4594-ab15-a89a2ad75b32 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:37.092863 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb10701-bc44-4594-ab15-a89a2ad75b32 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:37.095745 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb10701-bc44-4594-ab15-a89a2ad75b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:37.150751 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-cbb10701-bc44-4594-ab15-a89a2ad75b32 None None] MFA Rules not processed for user `e60a5e751f22450c8e366c85b37af1f6`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:37.159488 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-cbb10701-bc44-4594-ab15-a89a2ad75b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:37.217191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053963e7-9792-47f1-95b7-cc30c5858d49 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:37.234068 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea26a8a-cdae-4ad9-a16b-5814ceb98205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:37.237106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea26a8a-cdae-4ad9-a16b-5814ceb98205 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:37.237484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea26a8a-cdae-4ad9-a16b-5814ceb98205 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:37.237771 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea26a8a-cdae-4ad9-a16b-5814ceb98205 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:37.239357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea26a8a-cdae-4ad9-a16b-5814ceb98205 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:37.246760 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea26a8a-cdae-4ad9-a16b-5814ceb98205 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:37.315621 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 902/1817] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:37.322320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 916/1818] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:56:36 2022] POST /identity/v3/users => generated 290 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:56:37.327088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb10701-bc44-4594-ab15-a89a2ad75b32 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:37.344753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb10701-bc44-4594-ab15-a89a2ad75b32 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:37.351349 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db3355-ad89-4387-9633-398f2be8fbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:37.352319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db3355-ad89-4387-9633-398f2be8fbc9 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:37.352683 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db3355-ad89-4387-9633-398f2be8fbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:37.352963 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db3355-ad89-4387-9633-398f2be8fbc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:37.353795 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db3355-ad89-4387-9633-398f2be8fbc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:37.358110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db3355-ad89-4387-9633-398f2be8fbc9 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:37.372645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 903/1819] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:37.377289 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea26a8a-cdae-4ad9-a16b-5814ceb98205 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:37.391387 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea26a8a-cdae-4ad9-a16b-5814ceb98205 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:37.403132 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654982d-8fc3-40f5-b13d-694e30887caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:37.405622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654982d-8fc3-40f5-b13d-694e30887caa admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:37.405622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654982d-8fc3-40f5-b13d-694e30887caa admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:37.407721 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654982d-8fc3-40f5-b13d-694e30887caa admin admin] PATH_INFO: `/v3/projects/8bfe9508906043ac8c60e981f67e88e5/users/2d72867d46bb4865a53779ee45ae7b66/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:37.435201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654982d-8fc3-40f5-b13d-694e30887caa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bfe9508906043ac8c60e981f67e88e5, user_id=2d72867d46bb4865a53779ee45ae7b66, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:37.438141 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654982d-8fc3-40f5-b13d-694e30887caa admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:37.465819 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 917/1820] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:37 2022] PUT /identity/v3/projects/8bfe9508906043ac8c60e981f67e88e5/users/2d72867d46bb4865a53779ee45ae7b66/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:37.469902 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db3355-ad89-4387-9633-398f2be8fbc9 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:37.493508 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db3355-ad89-4387-9633-398f2be8fbc9 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:37.588388 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68d284-1af4-48f7-b5c5-e6db89063fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:37.591163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68d284-1af4-48f7-b5c5-e6db89063fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:37.592305 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68d284-1af4-48f7-b5c5-e6db89063fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:37.592740 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68d284-1af4-48f7-b5c5-e6db89063fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:37.594447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b68d284-1af4-48f7-b5c5-e6db89063fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:37.600833 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b68d284-1af4-48f7-b5c5-e6db89063fc8 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:37.615387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 904/1821] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:37.619814 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654982d-8fc3-40f5-b13d-694e30887caa admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:37.633698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654982d-8fc3-40f5-b13d-694e30887caa admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:37.642543 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d6456-74da-4675-97a1-5bba3b8bdd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:37.644756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d6456-74da-4675-97a1-5bba3b8bdd62 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:37.645860 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d6456-74da-4675-97a1-5bba3b8bdd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:37.645860 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d6456-74da-4675-97a1-5bba3b8bdd62 admin admin] PATH_INFO: `/v3/projects/8bfe9508906043ac8c60e981f67e88e5/users/2d72867d46bb4865a53779ee45ae7b66/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:37.648743 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d6456-74da-4675-97a1-5bba3b8bdd62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bfe9508906043ac8c60e981f67e88e5, user_id=2d72867d46bb4865a53779ee45ae7b66, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:37.654319 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d6456-74da-4675-97a1-5bba3b8bdd62 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:37.691337 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 918/1822] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:37 2022] PUT /identity/v3/projects/8bfe9508906043ac8c60e981f67e88e5/users/2d72867d46bb4865a53779ee45ae7b66/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:37.704329 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f6be1-495d-474c-846e-3f643f762a49 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:37.705095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f6be1-495d-474c-846e-3f643f762a49 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:37.705551 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f6be1-495d-474c-846e-3f643f762a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:37.738690 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-ad7f6be1-495d-474c-846e-3f643f762a49 None None] MFA Rules not processed for user `2d72867d46bb4865a53779ee45ae7b66`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:37.740336 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-ad7f6be1-495d-474c-846e-3f643f762a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:37.822168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:37.868218 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 905/1823] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:37 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:37.881500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d6456-74da-4675-97a1-5bba3b8bdd62 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:37.883429 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-695d6456-74da-4675-97a1-5bba3b8bdd62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:37.968462 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d6456-74da-4675-97a1-5bba3b8bdd62 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:37.972758 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619e08a-0217-45a6-b4d4-9f89ece6e25a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:37.973576 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619e08a-0217-45a6-b4d4-9f89ece6e25a service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:37.974037 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619e08a-0217-45a6-b4d4-9f89ece6e25a service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:37.974312 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619e08a-0217-45a6-b4d4-9f89ece6e25a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:37.986275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619e08a-0217-45a6-b4d4-9f89ece6e25a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:37.988681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619e08a-0217-45a6-b4d4-9f89ece6e25a service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:38.002059 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 919/1824] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:37 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:38.345738 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f6be1-495d-474c-846e-3f643f762a49 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:38.347896 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-ad7f6be1-495d-474c-846e-3f643f762a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:38.564709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f6be1-495d-474c-846e-3f643f762a49 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:38.574642 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a51d2-9805-4e48-9659-a3e201216841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:38.575919 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a51d2-9805-4e48-9659-a3e201216841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:38.576557 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a51d2-9805-4e48-9659-a3e201216841 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:38.577033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a51d2-9805-4e48-9659-a3e201216841 admin admin] PATH_INFO: `/v3/projects/3c9b886ab2b54d9a9751f08adda65517` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:38.578919 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a51d2-9805-4e48-9659-a3e201216841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c9b886ab2b54d9a9751f08adda65517)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:38.581286 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a51d2-9805-4e48-9659-a3e201216841 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:38.712133 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-229a51d2-9805-4e48-9659-a3e201216841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c9b886ab2b54d9a9751f08adda65517', 'request_id': 'req-229a51d2-9805-4e48-9659-a3e201216841'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:38.726806 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-229a51d2-9805-4e48-9659-a3e201216841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c9b886ab2b54d9a9751f08adda65517', 'request_id': 'req-229a51d2-9805-4e48-9659-a3e201216841'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:38.740507 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-229a51d2-9805-4e48-9659-a3e201216841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c9b886ab2b54d9a9751f08adda65517', 'request_id': 'req-229a51d2-9805-4e48-9659-a3e201216841'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:38.766547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-229a51d2-9805-4e48-9659-a3e201216841 admin admin] The token cache is being invalidate because project 3c9b886ab2b54d9a9751f08adda65517 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:38.766841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-229a51d2-9805-4e48-9659-a3e201216841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:38.768996 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-229a51d2-9805-4e48-9659-a3e201216841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:38.771985 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 906/1825] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:56:38 2022] DELETE /identity/v3/projects/3c9b886ab2b54d9a9751f08adda65517 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:38.777416 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619e08a-0217-45a6-b4d4-9f89ece6e25a service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:38.807990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619e08a-0217-45a6-b4d4-9f89ece6e25a service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:38.822210 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:39.024358 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4261f-b1c4-4049-92b4-718806c13823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:39.027952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4261f-b1c4-4049-92b4-718806c13823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:39.030041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4261f-b1c4-4049-92b4-718806c13823 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:39.032424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4261f-b1c4-4049-92b4-718806c13823 admin admin] PATH_INFO: `/v3/users/c1623f961a8b4002a26c50a14cdb85da` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:39.037476 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed4261f-b1c4-4049-92b4-718806c13823 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1623f961a8b4002a26c50a14cdb85da)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:39.045305 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed4261f-b1c4-4049-92b4-718806c13823 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:39.234434 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0ed4261f-b1c4-4049-92b4-718806c13823 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1623f961a8b4002a26c50a14cdb85da'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:39.243260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5dc01-ed0c-41d3-ae89-a74ce4b1dfad None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:39.243727 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5dc01-ed0c-41d3-ae89-a74ce4b1dfad None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:39.244106 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5dc01-ed0c-41d3-ae89-a74ce4b1dfad None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:39.248912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 907/1826] 10.222.0.78 () {54 vars in 921 bytes} [Wed May 25 14:56:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:56:39.300975 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0ed4261f-b1c4-4049-92b4-718806c13823 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1623f961a8b4002a26c50a14cdb85da'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:39.309157 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3ccbf-6c74-4e0a-84a0-daff85678f15 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:39.309609 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3ccbf-6c74-4e0a-84a0-daff85678f15 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:39.309989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3ccbf-6c74-4e0a-84a0-daff85678f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:39.342965 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0ed4261f-b1c4-4049-92b4-718806c13823 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1623f961a8b4002a26c50a14cdb85da'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:39.342965 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0ed4261f-b1c4-4049-92b4-718806c13823 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1623f961a8b4002a26c50a14cdb85da'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:39.343259 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0ed4261f-b1c4-4049-92b4-718806c13823 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1623f961a8b4002a26c50a14cdb85da'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:39.383418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-fce3ccbf-6c74-4e0a-84a0-daff85678f15 None None] MFA Rules not processed for user `e34e1d7ffbac49b0bde45829af0c6925`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:39.388793 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-fce3ccbf-6c74-4e0a-84a0-daff85678f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:39.409332 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 920/1827] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:56:38 2022] DELETE /identity/v3/users/c1623f961a8b4002a26c50a14cdb85da => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:39.676415 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 908/1828] 10.222.0.78 () {58 vars in 1018 bytes} [Wed May 25 14:56:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:40.035758 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4261f-b1c4-4049-92b4-718806c13823 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:40.037878 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-0ed4261f-b1c4-4049-92b4-718806c13823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:40.317943 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4261f-b1c4-4049-92b4-718806c13823 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:40.327357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ccbbe0-488a-4eff-8156-5bc0480c41b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:40.328048 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ccbbe0-488a-4eff-8156-5bc0480c41b0 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:40.328430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ccbbe0-488a-4eff-8156-5bc0480c41b0 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:40.328891 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ccbbe0-488a-4eff-8156-5bc0480c41b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:40.343828 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ccbbe0-488a-4eff-8156-5bc0480c41b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:40.346590 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ccbbe0-488a-4eff-8156-5bc0480c41b0 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:40.374220 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 921/1829] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:40.465115 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3ccbf-6c74-4e0a-84a0-daff85678f15 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:40.467202 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-fce3ccbf-6c74-4e0a-84a0-daff85678f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:40.497746 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3ccbf-6c74-4e0a-84a0-daff85678f15 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:40.504661 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:40.578178 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b545e82-9ca5-4ebe-a4c4-306c44ccda2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:40.579236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b545e82-9ca5-4ebe-a4c4-306c44ccda2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:40.580433 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b545e82-9ca5-4ebe-a4c4-306c44ccda2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:40.580433 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b545e82-9ca5-4ebe-a4c4-306c44ccda2f admin admin] PATH_INFO: `/v3/projects/8a73b10aa92d452a890c7d906b8303c3` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:40.581517 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b545e82-9ca5-4ebe-a4c4-306c44ccda2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a73b10aa92d452a890c7d906b8303c3)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:40.583805 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b545e82-9ca5-4ebe-a4c4-306c44ccda2f admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:40.670919 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-0b545e82-9ca5-4ebe-a4c4-306c44ccda2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a73b10aa92d452a890c7d906b8303c3', 'request_id': 'req-0b545e82-9ca5-4ebe-a4c4-306c44ccda2f'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:40.676544 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-0b545e82-9ca5-4ebe-a4c4-306c44ccda2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a73b10aa92d452a890c7d906b8303c3', 'request_id': 'req-0b545e82-9ca5-4ebe-a4c4-306c44ccda2f'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:40.684279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-0b545e82-9ca5-4ebe-a4c4-306c44ccda2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a73b10aa92d452a890c7d906b8303c3', 'request_id': 'req-0b545e82-9ca5-4ebe-a4c4-306c44ccda2f'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:40.701543 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-0b545e82-9ca5-4ebe-a4c4-306c44ccda2f admin admin] The token cache is being invalidate because project 8a73b10aa92d452a890c7d906b8303c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:40.701725 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-0b545e82-9ca5-4ebe-a4c4-306c44ccda2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:40.702398 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-0b545e82-9ca5-4ebe-a4c4-306c44ccda2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:40.704040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 909/1830] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:56:40 2022] DELETE /identity/v3/projects/8a73b10aa92d452a890c7d906b8303c3 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:40.709165 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a79edc-248c-4ef5-aa0e-fe6baeb8aceb None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:40.709487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a79edc-248c-4ef5-aa0e-fe6baeb8aceb None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:40.709786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a79edc-248c-4ef5-aa0e-fe6baeb8aceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:40.728862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-00a79edc-248c-4ef5-aa0e-fe6baeb8aceb None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:40.730133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-00a79edc-248c-4ef5-aa0e-fe6baeb8aceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:40.740350 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:40.811367 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b545e82-9ca5-4ebe-a4c4-306c44ccda2f admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:40.813406 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-0b545e82-9ca5-4ebe-a4c4-306c44ccda2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:40.822054 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 922/1831] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:40.827578 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b545e82-9ca5-4ebe-a4c4-306c44ccda2f admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:40.830552 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a09ed-8fc8-4055-aeb7-95d29e305609 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:40.831221 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a09ed-8fc8-4055-aeb7-95d29e305609 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:40.831846 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a09ed-8fc8-4055-aeb7-95d29e305609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:40.832676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567b34b-0def-4b1d-8b7b-4a0ae87f69d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:40.833844 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567b34b-0def-4b1d-8b7b-4a0ae87f69d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:40.834299 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567b34b-0def-4b1d-8b7b-4a0ae87f69d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:40.834650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567b34b-0def-4b1d-8b7b-4a0ae87f69d9 admin admin] PATH_INFO: `/v3/users/218760f94a944b9c8a7bc436f49d97f9` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:40.836058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567b34b-0def-4b1d-8b7b-4a0ae87f69d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=218760f94a944b9c8a7bc436f49d97f9)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:40.838044 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567b34b-0def-4b1d-8b7b-4a0ae87f69d9 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:40.857733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-6f4a09ed-8fc8-4055-aeb7-95d29e305609 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:40.859476 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-6f4a09ed-8fc8-4055-aeb7-95d29e305609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:40.870624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 923/1832] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:40.875687 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a09ed-8fc8-4055-aeb7-95d29e305609 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:40.886265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a09ed-8fc8-4055-aeb7-95d29e305609 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:40.893451 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560098f0-0a2b-44b5-934c-3c32ebbeb098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:40.894450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560098f0-0a2b-44b5-934c-3c32ebbeb098 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:40.894885 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560098f0-0a2b-44b5-934c-3c32ebbeb098 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:40.895584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560098f0-0a2b-44b5-934c-3c32ebbeb098 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:40.896471 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560098f0-0a2b-44b5-934c-3c32ebbeb098 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:56:40.897009 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560098f0-0a2b-44b5-934c-3c32ebbeb098 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:40.899563 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560098f0-0a2b-44b5-934c-3c32ebbeb098 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:40.919173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 924/1833] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:56:40.921801 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560098f0-0a2b-44b5-934c-3c32ebbeb098 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:40.921983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2567b34b-0def-4b1d-8b7b-4a0ae87f69d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '218760f94a944b9c8a7bc436f49d97f9'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:40.945064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2567b34b-0def-4b1d-8b7b-4a0ae87f69d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '218760f94a944b9c8a7bc436f49d97f9'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:40.951853 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560098f0-0a2b-44b5-934c-3c32ebbeb098 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:40.959687 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba26f87-3c01-4c46-8941-a23bde4a1ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:40.960513 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba26f87-3c01-4c46-8941-a23bde4a1ef2 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:40.960814 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba26f87-3c01-4c46-8941-a23bde4a1ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:40.961119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba26f87-3c01-4c46-8941-a23bde4a1ef2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:40.962157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba26f87-3c01-4c46-8941-a23bde4a1ef2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:40.964001 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2567b34b-0def-4b1d-8b7b-4a0ae87f69d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '218760f94a944b9c8a7bc436f49d97f9'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:40.964512 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba26f87-3c01-4c46-8941-a23bde4a1ef2 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:40.964870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2567b34b-0def-4b1d-8b7b-4a0ae87f69d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '218760f94a944b9c8a7bc436f49d97f9'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:40.965635 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2567b34b-0def-4b1d-8b7b-4a0ae87f69d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '218760f94a944b9c8a7bc436f49d97f9'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:40.991976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 910/1834] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:56:40 2022] DELETE /identity/v3/users/218760f94a944b9c8a7bc436f49d97f9 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:40.998421 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b697499-e28b-4352-8d13-d598d3330cc7 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:40.998850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b697499-e28b-4352-8d13-d598d3330cc7 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:40.999216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b697499-e28b-4352-8d13-d598d3330cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:41.009928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 925/1835] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:56:40 2022] POST /identity/v3/projects => generated 385 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:41.014928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba26f87-3c01-4c46-8941-a23bde4a1ef2 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:41.019102 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5ba26f87-3c01-4c46-8941-a23bde4a1ef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:41.024525 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-8b697499-e28b-4352-8d13-d598d3330cc7 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:41.025929 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-8b697499-e28b-4352-8d13-d598d3330cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:41.034060 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:41.198087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 911/1836] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:56:41.212434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b697499-e28b-4352-8d13-d598d3330cc7 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:41.214077 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-8b697499-e28b-4352-8d13-d598d3330cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:41.256867 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba26f87-3c01-4c46-8941-a23bde4a1ef2 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:41.277928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e4854-939a-44f4-9c26-2230d1188f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:41.278714 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e4854-939a-44f4-9c26-2230d1188f13 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:41.279005 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e4854-939a-44f4-9c26-2230d1188f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:41.279286 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e4854-939a-44f4-9c26-2230d1188f13 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:41.280024 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e4854-939a-44f4-9c26-2230d1188f13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:41.282117 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e4854-939a-44f4-9c26-2230d1188f13 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:41.320856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b697499-e28b-4352-8d13-d598d3330cc7 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:41.328846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b6912-57f2-4e7c-aa23-79b174d13765 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:41.330259 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b6912-57f2-4e7c-aa23-79b174d13765 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:41.331302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b6912-57f2-4e7c-aa23-79b174d13765 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:41.333081 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b6912-57f2-4e7c-aa23-79b174d13765 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:41.355757 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 926/1837] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:56:41 2022] POST /identity/v3/users => generated 293 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:41.356567 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b6912-57f2-4e7c-aa23-79b174d13765 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:41.357385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e4854-939a-44f4-9c26-2230d1188f13 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:41.359170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b6912-57f2-4e7c-aa23-79b174d13765 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:41.379709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 912/1838] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:56:41.390071 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e4854-939a-44f4-9c26-2230d1188f13 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:41.395885 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bad82f-6224-4c2e-bd7b-363423386e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:41.396644 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bad82f-6224-4c2e-bd7b-363423386e01 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:41.396989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bad82f-6224-4c2e-bd7b-363423386e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:41.397240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bad82f-6224-4c2e-bd7b-363423386e01 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:41.397981 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bad82f-6224-4c2e-bd7b-363423386e01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:41.399683 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bad82f-6224-4c2e-bd7b-363423386e01 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:41.424948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 927/1839] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:41.429224 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b6912-57f2-4e7c-aa23-79b174d13765 service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:41.440522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b6912-57f2-4e7c-aa23-79b174d13765 service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:41.455132 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80084ff9-dd94-404b-a92d-2ee427b24215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:41.455921 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80084ff9-dd94-404b-a92d-2ee427b24215 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:41.456213 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80084ff9-dd94-404b-a92d-2ee427b24215 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:41.457149 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80084ff9-dd94-404b-a92d-2ee427b24215 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:41.458201 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80084ff9-dd94-404b-a92d-2ee427b24215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:41.460193 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80084ff9-dd94-404b-a92d-2ee427b24215 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:41.529335 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 913/1840] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:41.534040 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bad82f-6224-4c2e-bd7b-363423386e01 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:41.548478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bad82f-6224-4c2e-bd7b-363423386e01 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:41.554075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62fc151-2ba9-4330-81f4-56c4096c74e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:41.555149 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fc151-2ba9-4330-81f4-56c4096c74e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:41.555688 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fc151-2ba9-4330-81f4-56c4096c74e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:41.556067 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fc151-2ba9-4330-81f4-56c4096c74e8 admin admin] PATH_INFO: `/v3/projects/b34a911130ca47d7a8ddd128027be294/users/c3e376f4538f43c196ebd68b0cf032fc/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:41.578357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62fc151-2ba9-4330-81f4-56c4096c74e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b34a911130ca47d7a8ddd128027be294, user_id=c3e376f4538f43c196ebd68b0cf032fc, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:41.582181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62fc151-2ba9-4330-81f4-56c4096c74e8 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:41.604015 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 928/1841] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:41 2022] PUT /identity/v3/projects/b34a911130ca47d7a8ddd128027be294/users/c3e376f4538f43c196ebd68b0cf032fc/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:41.615514 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a98d5-9b32-4424-b6d7-4281f5d3882d None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:41.615805 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a98d5-9b32-4424-b6d7-4281f5d3882d None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:41.616078 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a98d5-9b32-4424-b6d7-4281f5d3882d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:41.662761 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-2b0a98d5-9b32-4424-b6d7-4281f5d3882d None None] MFA Rules not processed for user `c3e376f4538f43c196ebd68b0cf032fc`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:41.664524 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-2b0a98d5-9b32-4424-b6d7-4281f5d3882d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:41.756066 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 914/1842] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:41.760327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62fc151-2ba9-4330-81f4-56c4096c74e8 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:41.811987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:41.857467 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62fc151-2ba9-4330-81f4-56c4096c74e8 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:41.863057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af642b77-6891-4c15-b157-f622d469577f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:41.863979 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af642b77-6891-4c15-b157-f622d469577f admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:41.864299 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af642b77-6891-4c15-b157-f622d469577f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:41.864594 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af642b77-6891-4c15-b157-f622d469577f admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:41.865837 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af642b77-6891-4c15-b157-f622d469577f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:41.868345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af642b77-6891-4c15-b157-f622d469577f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:41.911541 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 929/1843] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:56:41 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:41.915927 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a98d5-9b32-4424-b6d7-4281f5d3882d None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:41.933389 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a98d5-9b32-4424-b6d7-4281f5d3882d None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:41.988922 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:42.025091 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9494e5-ea25-44b9-8de5-30da5e49f929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:42.025789 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9494e5-ea25-44b9-8de5-30da5e49f929 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:42.026119 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9494e5-ea25-44b9-8de5-30da5e49f929 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:42.026765 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9494e5-ea25-44b9-8de5-30da5e49f929 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:42.027101 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9494e5-ea25-44b9-8de5-30da5e49f929 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:42.030900 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9494e5-ea25-44b9-8de5-30da5e49f929 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:42.067209 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 915/1844] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:56:41 2022] POST /identity/v3/users => generated 300 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:42.071445 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af642b77-6891-4c15-b157-f622d469577f admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:42.091177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af642b77-6891-4c15-b157-f622d469577f admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:42.106113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce9651-0ce9-4dbf-a388-dc9e56c5d48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:42.106875 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce9651-0ce9-4dbf-a388-dc9e56c5d48b admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:42.107170 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce9651-0ce9-4dbf-a388-dc9e56c5d48b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:42.107454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce9651-0ce9-4dbf-a388-dc9e56c5d48b admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:42.108209 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce9651-0ce9-4dbf-a388-dc9e56c5d48b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:42.109887 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce9651-0ce9-4dbf-a388-dc9e56c5d48b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:42.144639 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 930/1845] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:42.145439 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9494e5-ea25-44b9-8de5-30da5e49f929 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:42.159420 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9494e5-ea25-44b9-8de5-30da5e49f929 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:42.177663 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4025c4-a76c-4fcd-bac5-40279011584b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:42.178895 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4025c4-a76c-4fcd-bac5-40279011584b admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:42.183260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4025c4-a76c-4fcd-bac5-40279011584b admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:42.184627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4025c4-a76c-4fcd-bac5-40279011584b admin admin] PATH_INFO: `/v3/projects/ae4ce27b43104ebebe13ffaef7776f69/users/08c201126f8448d1880c5d77be87f92d/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:42.231219 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4025c4-a76c-4fcd-bac5-40279011584b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae4ce27b43104ebebe13ffaef7776f69, user_id=08c201126f8448d1880c5d77be87f92d, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:42.239667 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4025c4-a76c-4fcd-bac5-40279011584b admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:42.275397 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 916/1846] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:42 2022] PUT /identity/v3/projects/ae4ce27b43104ebebe13ffaef7776f69/users/08c201126f8448d1880c5d77be87f92d/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:56:42.279806 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce9651-0ce9-4dbf-a388-dc9e56c5d48b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:42.302367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce9651-0ce9-4dbf-a388-dc9e56c5d48b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:42.453284 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fd5ce-14c9-4a53-ad81-cde1785fca42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:42.453992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fd5ce-14c9-4a53-ad81-cde1785fca42 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:42.454310 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fd5ce-14c9-4a53-ad81-cde1785fca42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:42.454592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fd5ce-14c9-4a53-ad81-cde1785fca42 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:42.455338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fd5ce-14c9-4a53-ad81-cde1785fca42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:42.458667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fd5ce-14c9-4a53-ad81-cde1785fca42 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:42.494343 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 931/1847] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:42.499659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4025c4-a76c-4fcd-bac5-40279011584b admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:42.520846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4025c4-a76c-4fcd-bac5-40279011584b admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:42.543291 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946b07a-a203-467b-b871-8d632d3fa2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:42.547934 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946b07a-a203-467b-b871-8d632d3fa2f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:42.549979 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946b07a-a203-467b-b871-8d632d3fa2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:42.550491 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946b07a-a203-467b-b871-8d632d3fa2f9 admin admin] PATH_INFO: `/v3/projects/ae4ce27b43104ebebe13ffaef7776f69/users/08c201126f8448d1880c5d77be87f92d/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:42.555205 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7946b07a-a203-467b-b871-8d632d3fa2f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae4ce27b43104ebebe13ffaef7776f69, user_id=08c201126f8448d1880c5d77be87f92d, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:42.569466 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7946b07a-a203-467b-b871-8d632d3fa2f9 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:42.605240 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 917/1848] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:42 2022] PUT /identity/v3/projects/ae4ce27b43104ebebe13ffaef7776f69/users/08c201126f8448d1880c5d77be87f92d/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:56:42.617874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a24cb1-c943-458c-8238-584b992cf045 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:42.618353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a24cb1-c943-458c-8238-584b992cf045 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:42.618741 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a24cb1-c943-458c-8238-584b992cf045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:42.676387 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-c7a24cb1-c943-458c-8238-584b992cf045 None None] MFA Rules not processed for user `08c201126f8448d1880c5d77be87f92d`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:42.682445 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c7a24cb1-c943-458c-8238-584b992cf045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:42.717863 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946b07a-a203-467b-b871-8d632d3fa2f9 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:42.720114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-7946b07a-a203-467b-b871-8d632d3fa2f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:42.866956 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 932/1849] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:42.876381 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a24cb1-c943-458c-8238-584b992cf045 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:42.876660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946b07a-a203-467b-b871-8d632d3fa2f9 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:42.879178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c7a24cb1-c943-458c-8238-584b992cf045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:42.882116 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819990f-ccf7-41c7-a709-14e89d2211d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:42.882999 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819990f-ccf7-41c7-a709-14e89d2211d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:42.883275 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819990f-ccf7-41c7-a709-14e89d2211d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:42.883546 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819990f-ccf7-41c7-a709-14e89d2211d6 admin admin] PATH_INFO: `/v3/projects/101f6923287148ad91c9c23291237a8b` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:42.884744 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2819990f-ccf7-41c7-a709-14e89d2211d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=101f6923287148ad91c9c23291237a8b)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:42.886732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2819990f-ccf7-41c7-a709-14e89d2211d6 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:42.935275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:42.976887 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a24cb1-c943-458c-8238-584b992cf045 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:42.982094 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2819990f-ccf7-41c7-a709-14e89d2211d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '101f6923287148ad91c9c23291237a8b', 'request_id': 'req-2819990f-ccf7-41c7-a709-14e89d2211d6'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:42.988944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2819990f-ccf7-41c7-a709-14e89d2211d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '101f6923287148ad91c9c23291237a8b', 'request_id': 'req-2819990f-ccf7-41c7-a709-14e89d2211d6'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:42.997891 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2819990f-ccf7-41c7-a709-14e89d2211d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '101f6923287148ad91c9c23291237a8b', 'request_id': 'req-2819990f-ccf7-41c7-a709-14e89d2211d6'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:43.016127 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2819990f-ccf7-41c7-a709-14e89d2211d6 admin admin] The token cache is being invalidate because project 101f6923287148ad91c9c23291237a8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:43.016452 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2819990f-ccf7-41c7-a709-14e89d2211d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:43.017938 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2819990f-ccf7-41c7-a709-14e89d2211d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:43.018964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 918/1850] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:56:42 2022] DELETE /identity/v3/projects/101f6923287148ad91c9c23291237a8b => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:56:43.023846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819990f-ccf7-41c7-a709-14e89d2211d6 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:43.025420 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-2819990f-ccf7-41c7-a709-14e89d2211d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:43.043603 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7d010-4ddd-4641-894d-6d69c8af9e5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:43.044193 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7d010-4ddd-4641-894d-6d69c8af9e5e service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:43.044530 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7d010-4ddd-4641-894d-6d69c8af9e5e service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:43.045252 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7d010-4ddd-4641-894d-6d69c8af9e5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:43.047025 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9ea7d010-4ddd-4641-894d-6d69c8af9e5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:43.053236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:43.116311 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819990f-ccf7-41c7-a709-14e89d2211d6 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:43.123434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753b6d7-5d98-4d04-a2d0-61a6d02bf1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:43.124943 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753b6d7-5d98-4d04-a2d0-61a6d02bf1df admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:43.125243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753b6d7-5d98-4d04-a2d0-61a6d02bf1df admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:43.125551 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753b6d7-5d98-4d04-a2d0-61a6d02bf1df admin admin] PATH_INFO: `/v3/users/2d72867d46bb4865a53779ee45ae7b66` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:43.127026 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753b6d7-5d98-4d04-a2d0-61a6d02bf1df admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d72867d46bb4865a53779ee45ae7b66)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:43.129165 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753b6d7-5d98-4d04-a2d0-61a6d02bf1df admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:43.139899 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7d010-4ddd-4641-894d-6d69c8af9e5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:43.141404 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7d010-4ddd-4641-894d-6d69c8af9e5e service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:43.156460 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 933/1851] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:43.203190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7d010-4ddd-4641-894d-6d69c8af9e5e service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:43.204978 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9ea7d010-4ddd-4641-894d-6d69c8af9e5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:43.212725 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c753b6d7-5d98-4d04-a2d0-61a6d02bf1df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d72867d46bb4865a53779ee45ae7b66'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:43.220424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7d010-4ddd-4641-894d-6d69c8af9e5e service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:43.225323 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2182e1-aa7d-4c79-81c1-a199109a24f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:43.226457 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2182e1-aa7d-4c79-81c1-a199109a24f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:43.226804 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2182e1-aa7d-4c79-81c1-a199109a24f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:43.227136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2182e1-aa7d-4c79-81c1-a199109a24f6 admin admin] PATH_INFO: `/v3/users/c3e376f4538f43c196ebd68b0cf032fc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:43.227821 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c753b6d7-5d98-4d04-a2d0-61a6d02bf1df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d72867d46bb4865a53779ee45ae7b66'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:43.228627 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2182e1-aa7d-4c79-81c1-a199109a24f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3e376f4538f43c196ebd68b0cf032fc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:43.231669 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2182e1-aa7d-4c79-81c1-a199109a24f6 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:43.238841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c753b6d7-5d98-4d04-a2d0-61a6d02bf1df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d72867d46bb4865a53779ee45ae7b66'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:43.239598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c753b6d7-5d98-4d04-a2d0-61a6d02bf1df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d72867d46bb4865a53779ee45ae7b66'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:43.240308 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c753b6d7-5d98-4d04-a2d0-61a6d02bf1df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d72867d46bb4865a53779ee45ae7b66'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:43.257061 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 919/1852] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:56:43 2022] DELETE /identity/v3/users/2d72867d46bb4865a53779ee45ae7b66 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:43.290509 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fc2182e1-aa7d-4c79-81c1-a199109a24f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3e376f4538f43c196ebd68b0cf032fc'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:43.305666 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fc2182e1-aa7d-4c79-81c1-a199109a24f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3e376f4538f43c196ebd68b0cf032fc'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:43.320070 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fc2182e1-aa7d-4c79-81c1-a199109a24f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3e376f4538f43c196ebd68b0cf032fc'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:43.321130 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fc2182e1-aa7d-4c79-81c1-a199109a24f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3e376f4538f43c196ebd68b0cf032fc'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:43.322361 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fc2182e1-aa7d-4c79-81c1-a199109a24f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3e376f4538f43c196ebd68b0cf032fc'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:43.337207 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:43.342848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 934/1853] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:56:43 2022] DELETE /identity/v3/users/c3e376f4538f43c196ebd68b0cf032fc => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:43.348653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757829c-2011-4818-82f1-82010b46319e None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:43.349024 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757829c-2011-4818-82f1-82010b46319e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:43.349307 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757829c-2011-4818-82f1-82010b46319e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:43.372510 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-8757829c-2011-4818-82f1-82010b46319e None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:43.374126 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-8757829c-2011-4818-82f1-82010b46319e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:43.455735 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 920/1854] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:43.465521 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2182e1-aa7d-4c79-81c1-a199109a24f6 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:43.466890 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-fc2182e1-aa7d-4c79-81c1-a199109a24f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:43.544161 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2182e1-aa7d-4c79-81c1-a199109a24f6 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:43.550494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32e6f4-47a7-45f3-bf5d-bc16b06d99b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:43.551416 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32e6f4-47a7-45f3-bf5d-bc16b06d99b3 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:43.551707 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32e6f4-47a7-45f3-bf5d-bc16b06d99b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:43.552016 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32e6f4-47a7-45f3-bf5d-bc16b06d99b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:43.566661 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32e6f4-47a7-45f3-bf5d-bc16b06d99b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:43.568284 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32e6f4-47a7-45f3-bf5d-bc16b06d99b3 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:43.595020 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 935/1855] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:43.610798 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757829c-2011-4818-82f1-82010b46319e None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:43.612350 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-8757829c-2011-4818-82f1-82010b46319e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:43.626840 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757829c-2011-4818-82f1-82010b46319e None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:43.634827 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aed1d6-1b9d-4f32-9aa4-1e88c8f792bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:43.636344 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aed1d6-1b9d-4f32-9aa4-1e88c8f792bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:43.637353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aed1d6-1b9d-4f32-9aa4-1e88c8f792bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:43.637836 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aed1d6-1b9d-4f32-9aa4-1e88c8f792bb admin admin] PATH_INFO: `/v3/projects/8bfe9508906043ac8c60e981f67e88e5` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:43.640019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aed1d6-1b9d-4f32-9aa4-1e88c8f792bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bfe9508906043ac8c60e981f67e88e5)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:43.643805 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aed1d6-1b9d-4f32-9aa4-1e88c8f792bb admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:43.719990 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:43.753509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d9aed1d6-1b9d-4f32-9aa4-1e88c8f792bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bfe9508906043ac8c60e981f67e88e5', 'request_id': 'req-d9aed1d6-1b9d-4f32-9aa4-1e88c8f792bb'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:43.770628 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d9aed1d6-1b9d-4f32-9aa4-1e88c8f792bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bfe9508906043ac8c60e981f67e88e5', 'request_id': 'req-d9aed1d6-1b9d-4f32-9aa4-1e88c8f792bb'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:43.785421 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d9aed1d6-1b9d-4f32-9aa4-1e88c8f792bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bfe9508906043ac8c60e981f67e88e5', 'request_id': 'req-d9aed1d6-1b9d-4f32-9aa4-1e88c8f792bb'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:43.807268 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d9aed1d6-1b9d-4f32-9aa4-1e88c8f792bb admin admin] The token cache is being invalidate because project 8bfe9508906043ac8c60e981f67e88e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:43.807783 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d9aed1d6-1b9d-4f32-9aa4-1e88c8f792bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:43.809638 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d9aed1d6-1b9d-4f32-9aa4-1e88c8f792bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:43.813088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 921/1856] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:56:43 2022] DELETE /identity/v3/projects/8bfe9508906043ac8c60e981f67e88e5 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:43.817719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d3360-c0af-4b9b-9773-dc88e1a1a9da None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:43.818032 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d3360-c0af-4b9b-9773-dc88e1a1a9da None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:43.818314 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d3360-c0af-4b9b-9773-dc88e1a1a9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:43.847758 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-1d2d3360-c0af-4b9b-9773-dc88e1a1a9da None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:43.849120 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1d2d3360-c0af-4b9b-9773-dc88e1a1a9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:43.929228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 936/1857] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:56:43.937549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524d0fc-35e8-430c-9176-15cb567eea40 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:43.937833 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524d0fc-35e8-430c-9176-15cb567eea40 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:43.938178 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524d0fc-35e8-430c-9176-15cb567eea40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:43.958197 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-e524d0fc-35e8-430c-9176-15cb567eea40 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:43.959483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-e524d0fc-35e8-430c-9176-15cb567eea40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:43.968053 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 922/1858] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:43.973339 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d3360-c0af-4b9b-9773-dc88e1a1a9da None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:43.983799 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d3360-c0af-4b9b-9773-dc88e1a1a9da None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:43.989624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d624c1-ca85-49c1-9185-4697230bcbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:43.989624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d624c1-ca85-49c1-9185-4697230bcbea admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:43.989887 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d624c1-ca85-49c1-9185-4697230bcbea admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:43.989956 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d624c1-ca85-49c1-9185-4697230bcbea admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:43.990697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d624c1-ca85-49c1-9185-4697230bcbea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:56:43.991008 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d624c1-ca85-49c1-9185-4697230bcbea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:43.992770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d624c1-ca85-49c1-9185-4697230bcbea admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:44.011438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 937/1859] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:56:44.013262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e524d0fc-35e8-430c-9176-15cb567eea40 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:44.024531 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e524d0fc-35e8-430c-9176-15cb567eea40 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:44.030504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcfb348-0987-41cc-8a96-a55d1bbd1dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:44.031255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcfb348-0987-41cc-8a96-a55d1bbd1dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:44.031598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcfb348-0987-41cc-8a96-a55d1bbd1dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:44.031975 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcfb348-0987-41cc-8a96-a55d1bbd1dc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:44.032984 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcfb348-0987-41cc-8a96-a55d1bbd1dc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:44.035186 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcfb348-0987-41cc-8a96-a55d1bbd1dc1 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:44.042155 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d624c1-ca85-49c1-9185-4697230bcbea admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:44.044107 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-89d624c1-ca85-49c1-9185-4697230bcbea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:44.060698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d624c1-ca85-49c1-9185-4697230bcbea admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:44.066960 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e25ed0-1c84-4575-959c-837ae20cf6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:44.067706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e25ed0-1c84-4575-959c-837ae20cf6f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:44.067990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e25ed0-1c84-4575-959c-837ae20cf6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:44.068353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e25ed0-1c84-4575-959c-837ae20cf6f8 admin admin] PATH_INFO: `/v3/projects/b34a911130ca47d7a8ddd128027be294` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:44.070590 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e25ed0-1c84-4575-959c-837ae20cf6f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b34a911130ca47d7a8ddd128027be294)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:44.072832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e25ed0-1c84-4575-959c-837ae20cf6f8 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:44.088796 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 923/1860] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:56:44 2022] POST /identity/v3/projects => generated 401 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:56:44.092656 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcfb348-0987-41cc-8a96-a55d1bbd1dc1 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:44.139965 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-46e25ed0-1c84-4575-959c-837ae20cf6f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b34a911130ca47d7a8ddd128027be294', 'request_id': 'req-46e25ed0-1c84-4575-959c-837ae20cf6f8'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:44.148006 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-46e25ed0-1c84-4575-959c-837ae20cf6f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b34a911130ca47d7a8ddd128027be294', 'request_id': 'req-46e25ed0-1c84-4575-959c-837ae20cf6f8'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:44.159401 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-46e25ed0-1c84-4575-959c-837ae20cf6f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b34a911130ca47d7a8ddd128027be294', 'request_id': 'req-46e25ed0-1c84-4575-959c-837ae20cf6f8'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:44.178469 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcfb348-0987-41cc-8a96-a55d1bbd1dc1 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:44.179525 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-46e25ed0-1c84-4575-959c-837ae20cf6f8 admin admin] The token cache is being invalidate because project b34a911130ca47d7a8ddd128027be294 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:44.179825 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-46e25ed0-1c84-4575-959c-837ae20cf6f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:44.180527 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-46e25ed0-1c84-4575-959c-837ae20cf6f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:44.182549 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 938/1861] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:56:44 2022] DELETE /identity/v3/projects/b34a911130ca47d7a8ddd128027be294 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:56:44.187224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e25ed0-1c84-4575-959c-837ae20cf6f8 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:44.188574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-46e25ed0-1c84-4575-959c-837ae20cf6f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:44.203577 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:44.246663 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0330e36-c6f1-418b-a772-6d0831c85a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:44.247444 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0330e36-c6f1-418b-a772-6d0831c85a4c admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:44.247796 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0330e36-c6f1-418b-a772-6d0831c85a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:44.248148 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0330e36-c6f1-418b-a772-6d0831c85a4c admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:44.249008 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0330e36-c6f1-418b-a772-6d0831c85a4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:44.251109 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0330e36-c6f1-418b-a772-6d0831c85a4c admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:44.267199 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e25ed0-1c84-4575-959c-837ae20cf6f8 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:44.273201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d3e5c-0d83-4144-a516-80f2f7606893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:44.274194 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d3e5c-0d83-4144-a516-80f2f7606893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:44.274553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d3e5c-0d83-4144-a516-80f2f7606893 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:44.274819 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d3e5c-0d83-4144-a516-80f2f7606893 admin admin] PATH_INFO: `/v3/users/08c201126f8448d1880c5d77be87f92d` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:44.279183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 924/1862] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:56:44 2022] POST /identity/v3/users => generated 301 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:56:44.279422 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d3e5c-0d83-4144-a516-80f2f7606893 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08c201126f8448d1880c5d77be87f92d)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:44.279422 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d3e5c-0d83-4144-a516-80f2f7606893 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:44.280293 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0330e36-c6f1-418b-a772-6d0831c85a4c admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:44.282033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-a0330e36-c6f1-418b-a772-6d0831c85a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:44.295565 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0330e36-c6f1-418b-a772-6d0831c85a4c admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:44.301013 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359840fc-1b2f-4080-bb17-2b9d2d81e76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:44.301735 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359840fc-1b2f-4080-bb17-2b9d2d81e76a admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:44.302036 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359840fc-1b2f-4080-bb17-2b9d2d81e76a admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:44.302349 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359840fc-1b2f-4080-bb17-2b9d2d81e76a admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:44.303139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359840fc-1b2f-4080-bb17-2b9d2d81e76a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:44.304859 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359840fc-1b2f-4080-bb17-2b9d2d81e76a admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:44.316905 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 925/1863] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:56:44.323089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359840fc-1b2f-4080-bb17-2b9d2d81e76a admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:44.334375 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359840fc-1b2f-4080-bb17-2b9d2d81e76a admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:44.340643 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbfa93-9c07-40f3-96de-b36cdecafd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:44.341319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbfa93-9c07-40f3-96de-b36cdecafd1d admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:44.341601 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbfa93-9c07-40f3-96de-b36cdecafd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:44.341875 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbfa93-9c07-40f3-96de-b36cdecafd1d admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:44.342537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-be9d3e5c-0d83-4144-a516-80f2f7606893 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08c201126f8448d1880c5d77be87f92d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:44.343010 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cbfa93-9c07-40f3-96de-b36cdecafd1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:44.344451 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cbfa93-9c07-40f3-96de-b36cdecafd1d admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:44.355973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 926/1864] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:56:44.356307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-be9d3e5c-0d83-4144-a516-80f2f7606893 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08c201126f8448d1880c5d77be87f92d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:44.360854 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbfa93-9c07-40f3-96de-b36cdecafd1d admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:44.368142 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-be9d3e5c-0d83-4144-a516-80f2f7606893 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08c201126f8448d1880c5d77be87f92d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:44.368366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:44.369137 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-be9d3e5c-0d83-4144-a516-80f2f7606893 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08c201126f8448d1880c5d77be87f92d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:44.369923 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-be9d3e5c-0d83-4144-a516-80f2f7606893 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08c201126f8448d1880c5d77be87f92d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:44.376310 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbfa93-9c07-40f3-96de-b36cdecafd1d admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:44.383691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037917b-65b9-4959-bc05-67c24581f129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:44.384838 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037917b-65b9-4959-bc05-67c24581f129 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:44.385378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037917b-65b9-4959-bc05-67c24581f129 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:44.385657 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037917b-65b9-4959-bc05-67c24581f129 admin admin] PATH_INFO: `/v3/projects/5aacf2979fb34111a51fbaf0f0034b1c/users/4ff7b61a377749e0bc319b66097fd58b/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:44.385928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 939/1865] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:56:44 2022] DELETE /identity/v3/users/08c201126f8448d1880c5d77be87f92d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:56:44.410147 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037917b-65b9-4959-bc05-67c24581f129 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5aacf2979fb34111a51fbaf0f0034b1c, user_id=4ff7b61a377749e0bc319b66097fd58b, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:44.413011 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037917b-65b9-4959-bc05-67c24581f129 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:44.430002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 927/1866] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:44 2022] PUT /identity/v3/projects/5aacf2979fb34111a51fbaf0f0034b1c/users/4ff7b61a377749e0bc319b66097fd58b/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:44.442845 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6c082-ca60-42df-ba97-3521da5c8351 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:44.443313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6c082-ca60-42df-ba97-3521da5c8351 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:44.443631 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6c082-ca60-42df-ba97-3521da5c8351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:44.478117 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-81a6c082-ca60-42df-ba97-3521da5c8351 None None] MFA Rules not processed for user `4ff7b61a377749e0bc319b66097fd58b`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:44.484777 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-81a6c082-ca60-42df-ba97-3521da5c8351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:44.563703 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 940/1867] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:44.567986 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037917b-65b9-4959-bc05-67c24581f129 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:44.569725 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-8037917b-65b9-4959-bc05-67c24581f129 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:44.647422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037917b-65b9-4959-bc05-67c24581f129 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:44.652849 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffa9e1-62e6-444c-9746-b51408ddc584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:44.653641 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffa9e1-62e6-444c-9746-b51408ddc584 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:44.653926 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffa9e1-62e6-444c-9746-b51408ddc584 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:44.654232 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffa9e1-62e6-444c-9746-b51408ddc584 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:44.654945 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ffa9e1-62e6-444c-9746-b51408ddc584 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:44.656967 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ffa9e1-62e6-444c-9746-b51408ddc584 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:44.687250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 928/1868] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:56:44 2022] POST /identity/v3/projects => generated 397 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:44.691579 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6c082-ca60-42df-ba97-3521da5c8351 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:44.703546 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6c082-ca60-42df-ba97-3521da5c8351 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:44.725432 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffa9e1-62e6-444c-9746-b51408ddc584 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:44.727535 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-15ffa9e1-62e6-444c-9746-b51408ddc584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:44.731495 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:44.790031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29548a85-c8b8-4b1d-bc9e-f1afc318c07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:44.790782 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29548a85-c8b8-4b1d-bc9e-f1afc318c07e admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:44.791123 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29548a85-c8b8-4b1d-bc9e-f1afc318c07e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:44.791410 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29548a85-c8b8-4b1d-bc9e-f1afc318c07e admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:44.792136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29548a85-c8b8-4b1d-bc9e-f1afc318c07e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:44.795357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29548a85-c8b8-4b1d-bc9e-f1afc318c07e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:44.826497 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 941/1869] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:56:44 2022] POST /identity/v3/users => generated 305 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:44.831230 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29548a85-c8b8-4b1d-bc9e-f1afc318c07e admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:44.832990 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffa9e1-62e6-444c-9746-b51408ddc584 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:44.843087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910f284-b1f3-4121-8c12-dcea6e0f41bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:44.843274 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29548a85-c8b8-4b1d-bc9e-f1afc318c07e admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:44.843930 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910f284-b1f3-4121-8c12-dcea6e0f41bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:44.844234 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910f284-b1f3-4121-8c12-dcea6e0f41bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:44.844513 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910f284-b1f3-4121-8c12-dcea6e0f41bd admin admin] PATH_INFO: `/v3/projects/ae4ce27b43104ebebe13ffaef7776f69` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:44.846229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910f284-b1f3-4121-8c12-dcea6e0f41bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae4ce27b43104ebebe13ffaef7776f69)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:44.848207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910f284-b1f3-4121-8c12-dcea6e0f41bd admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:44.849285 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1ddb1-89fa-4688-a380-8aa9d6e373bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:44.849892 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1ddb1-89fa-4688-a380-8aa9d6e373bb admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:44.850241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1ddb1-89fa-4688-a380-8aa9d6e373bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:44.850526 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1ddb1-89fa-4688-a380-8aa9d6e373bb admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:44.851313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da1ddb1-89fa-4688-a380-8aa9d6e373bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:44.853096 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da1ddb1-89fa-4688-a380-8aa9d6e373bb admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:44.865915 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 942/1870] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:56:44.870259 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1ddb1-89fa-4688-a380-8aa9d6e373bb admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:44.880940 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1ddb1-89fa-4688-a380-8aa9d6e373bb admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:44.886374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd749d2c-66f6-472c-ab33-19806f2b8f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:44.888080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd749d2c-66f6-472c-ab33-19806f2b8f81 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:44.888365 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd749d2c-66f6-472c-ab33-19806f2b8f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:44.888642 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd749d2c-66f6-472c-ab33-19806f2b8f81 admin admin] PATH_INFO: `/v3/projects/2e187010915e4ea6a492217b229c8916/users/9ee7927c702b456e9a3ec1a286c8f6e4/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:44.905225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2910f284-b1f3-4121-8c12-dcea6e0f41bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae4ce27b43104ebebe13ffaef7776f69', 'request_id': 'req-2910f284-b1f3-4121-8c12-dcea6e0f41bd'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:44.912430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd749d2c-66f6-472c-ab33-19806f2b8f81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e187010915e4ea6a492217b229c8916, user_id=9ee7927c702b456e9a3ec1a286c8f6e4, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:44.914325 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2910f284-b1f3-4121-8c12-dcea6e0f41bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae4ce27b43104ebebe13ffaef7776f69', 'request_id': 'req-2910f284-b1f3-4121-8c12-dcea6e0f41bd'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:44.915526 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd749d2c-66f6-472c-ab33-19806f2b8f81 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:44.922936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2910f284-b1f3-4121-8c12-dcea6e0f41bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae4ce27b43104ebebe13ffaef7776f69', 'request_id': 'req-2910f284-b1f3-4121-8c12-dcea6e0f41bd'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:44.928118 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:44.941613 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2910f284-b1f3-4121-8c12-dcea6e0f41bd admin admin] The token cache is being invalidate because project ae4ce27b43104ebebe13ffaef7776f69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:44.943094 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2910f284-b1f3-4121-8c12-dcea6e0f41bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:44.943842 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2910f284-b1f3-4121-8c12-dcea6e0f41bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:44.946945 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 929/1871] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:56:44 2022] DELETE /identity/v3/projects/ae4ce27b43104ebebe13ffaef7776f69 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:44.951846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d1a72-1fb6-456f-a625-5a19d316f44e None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:44.952037 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 943/1872] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:44 2022] PUT /identity/v3/projects/2e187010915e4ea6a492217b229c8916/users/9ee7927c702b456e9a3ec1a286c8f6e4/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:56:44.952815 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d1a72-1fb6-456f-a625-5a19d316f44e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:44.953995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d1a72-1fb6-456f-a625-5a19d316f44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:44.955707 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd749d2c-66f6-472c-ab33-19806f2b8f81 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:44.957913 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-cd749d2c-66f6-472c-ab33-19806f2b8f81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:44.980927 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-7d8d1a72-1fb6-456f-a625-5a19d316f44e None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:44.982515 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-7d8d1a72-1fb6-456f-a625-5a19d316f44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:45.066590 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd749d2c-66f6-472c-ab33-19806f2b8f81 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:45.072394 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733944d-2154-4b26-a12e-1e093d3a903f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:45.073091 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733944d-2154-4b26-a12e-1e093d3a903f admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:45.073384 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733944d-2154-4b26-a12e-1e093d3a903f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:45.073666 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733944d-2154-4b26-a12e-1e093d3a903f admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:45.074680 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733944d-2154-4b26-a12e-1e093d3a903f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:45.076455 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733944d-2154-4b26-a12e-1e093d3a903f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:45.081560 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 930/1873] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:56:45.088352 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 944/1874] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:56:45.092028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf0868-df63-49c8-bb93-40174a8cb9d2 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:45.092328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf0868-df63-49c8-bb93-40174a8cb9d2 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:45.093724 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733944d-2154-4b26-a12e-1e093d3a903f admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:45.094441 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf0868-df63-49c8-bb93-40174a8cb9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:45.108310 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733944d-2154-4b26-a12e-1e093d3a903f admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:45.113382 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8c94a-d584-457d-8945-e6fa4789c797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:45.114360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8c94a-d584-457d-8945-e6fa4789c797 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:45.114616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8c94a-d584-457d-8945-e6fa4789c797 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:45.114925 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8c94a-d584-457d-8945-e6fa4789c797 admin admin] PATH_INFO: `/v3/projects/2e187010915e4ea6a492217b229c8916/users/9ee7927c702b456e9a3ec1a286c8f6e4/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:45.115096 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-24cf0868-df63-49c8-bb93-40174a8cb9d2 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:45.116312 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-24cf0868-df63-49c8-bb93-40174a8cb9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:45.116937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8c94a-d584-457d-8945-e6fa4789c797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e187010915e4ea6a492217b229c8916, user_id=9ee7927c702b456e9a3ec1a286c8f6e4, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:45.119725 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8c94a-d584-457d-8945-e6fa4789c797 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:45.126721 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 931/1875] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:56:45.133013 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 945/1876] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:45 2022] PUT /identity/v3/projects/2e187010915e4ea6a492217b229c8916/users/9ee7927c702b456e9a3ec1a286c8f6e4/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:56:45.134327 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf0868-df63-49c8-bb93-40174a8cb9d2 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:45.142104 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579988d-0b49-43e3-bd49-f980f7b87ff8 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:45.142458 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579988d-0b49-43e3-bd49-f980f7b87ff8 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:45.142667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579988d-0b49-43e3-bd49-f980f7b87ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:45.145868 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf0868-df63-49c8-bb93-40174a8cb9d2 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:45.173319 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-f579988d-0b49-43e3-bd49-f980f7b87ff8 None None] MFA Rules not processed for user `9ee7927c702b456e9a3ec1a286c8f6e4`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:45.175499 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f579988d-0b49-43e3-bd49-f980f7b87ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:45.239636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f5352-7a07-40f8-9b25-de856dfd6ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:45.240325 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f5352-7a07-40f8-9b25-de856dfd6ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:45.240613 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f5352-7a07-40f8-9b25-de856dfd6ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:45.240884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f5352-7a07-40f8-9b25-de856dfd6ce6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:45.241915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f5352-7a07-40f8-9b25-de856dfd6ce6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:56:45.241915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f5352-7a07-40f8-9b25-de856dfd6ce6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:45.243720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f5352-7a07-40f8-9b25-de856dfd6ce6 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:45.256710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:45.260888 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 932/1877] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:56:45.261730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f5352-7a07-40f8-9b25-de856dfd6ce6 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:45.277650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f5352-7a07-40f8-9b25-de856dfd6ce6 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:45.283918 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1961824-0845-4ecd-9d08-6126d5acc9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:45.284978 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1961824-0845-4ecd-9d08-6126d5acc9af admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:45.285280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1961824-0845-4ecd-9d08-6126d5acc9af admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:45.285611 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1961824-0845-4ecd-9d08-6126d5acc9af admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:45.286635 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1961824-0845-4ecd-9d08-6126d5acc9af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:45.288755 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1961824-0845-4ecd-9d08-6126d5acc9af admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:45.292126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 946/1878] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:45 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:56:45.304559 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579988d-0b49-43e3-bd49-f980f7b87ff8 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:45.306293 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f579988d-0b49-43e3-bd49-f980f7b87ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:45.336852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 933/1879] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:56:45 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:56:45.338323 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1961824-0845-4ecd-9d08-6126d5acc9af admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:45.410404 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579988d-0b49-43e3-bd49-f980f7b87ff8 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:45.426620 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1961824-0845-4ecd-9d08-6126d5acc9af admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:45.431328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a03688-31f5-424d-a1c5-b3f2eb7e0d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:45.431993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a03688-31f5-424d-a1c5-b3f2eb7e0d11 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:45.432277 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a03688-31f5-424d-a1c5-b3f2eb7e0d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:45.432551 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a03688-31f5-424d-a1c5-b3f2eb7e0d11 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:45.433231 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a03688-31f5-424d-a1c5-b3f2eb7e0d11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:45.435237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a03688-31f5-424d-a1c5-b3f2eb7e0d11 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:45.442994 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:45.457376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 934/1880] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:56:45 2022] POST /identity/v3/users => generated 292 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:45.462559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a03688-31f5-424d-a1c5-b3f2eb7e0d11 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:45.468849 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533dcfd-d7c7-4669-9ff8-3ec94fb1e9c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:45.469504 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533dcfd-d7c7-4669-9ff8-3ec94fb1e9c7 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:45.469786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533dcfd-d7c7-4669-9ff8-3ec94fb1e9c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:45.470090 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533dcfd-d7c7-4669-9ff8-3ec94fb1e9c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:45.473148 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a03688-31f5-424d-a1c5-b3f2eb7e0d11 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:45.478257 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ddf504-019e-400d-a48d-255742376639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:45.478930 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddf504-019e-400d-a48d-255742376639 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:45.479199 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddf504-019e-400d-a48d-255742376639 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:45.479474 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddf504-019e-400d-a48d-255742376639 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:45.480253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ddf504-019e-400d-a48d-255742376639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:45.482393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ddf504-019e-400d-a48d-255742376639 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:45.493354 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 935/1881] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:56:45.499995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ddf504-019e-400d-a48d-255742376639 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:45.509987 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ddf504-019e-400d-a48d-255742376639 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:45.514750 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00226b9b-a300-4b7f-9c5a-f27ca33035b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:45.515410 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00226b9b-a300-4b7f-9c5a-f27ca33035b9 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:45.515693 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00226b9b-a300-4b7f-9c5a-f27ca33035b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:45.515966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00226b9b-a300-4b7f-9c5a-f27ca33035b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:45.516692 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00226b9b-a300-4b7f-9c5a-f27ca33035b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:45.518662 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00226b9b-a300-4b7f-9c5a-f27ca33035b9 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:45.530774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 936/1882] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:45.535156 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00226b9b-a300-4b7f-9c5a-f27ca33035b9 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:45.545779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9533dcfd-d7c7-4669-9ff8-3ec94fb1e9c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:45.545985 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00226b9b-a300-4b7f-9c5a-f27ca33035b9 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:45.547306 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9533dcfd-d7c7-4669-9ff8-3ec94fb1e9c7 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:45.551653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0637c3-915b-41f9-98da-27d55e2ff57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:45.552541 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0637c3-915b-41f9-98da-27d55e2ff57a admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:45.552821 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0637c3-915b-41f9-98da-27d55e2ff57a admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:45.553095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0637c3-915b-41f9-98da-27d55e2ff57a admin admin] PATH_INFO: `/v3/projects/f2a776b2cf004d6eb1b77d6801208259/users/c40946dc499e497aa717c8eb7278b30d/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:45.557220 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 947/1883] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:45 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:56:45.571718 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0637c3-915b-41f9-98da-27d55e2ff57a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2a776b2cf004d6eb1b77d6801208259, user_id=c40946dc499e497aa717c8eb7278b30d, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:45.574525 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0637c3-915b-41f9-98da-27d55e2ff57a admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:45.595869 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 937/1884] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:45 2022] PUT /identity/v3/projects/f2a776b2cf004d6eb1b77d6801208259/users/c40946dc499e497aa717c8eb7278b30d/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:56:45.606893 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021f79e-57e7-44b5-8532-c74ee5c92b8a None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:45.607189 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021f79e-57e7-44b5-8532-c74ee5c92b8a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:45.607486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021f79e-57e7-44b5-8532-c74ee5c92b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:45.635439 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-7021f79e-57e7-44b5-8532-c74ee5c92b8a None None] MFA Rules not processed for user `c40946dc499e497aa717c8eb7278b30d`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:45.637198 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7021f79e-57e7-44b5-8532-c74ee5c92b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:45.651691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0637c3-915b-41f9-98da-27d55e2ff57a admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:45.684459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:45.711492 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0637c3-915b-41f9-98da-27d55e2ff57a admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:45.712302 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 948/1885] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:45.716697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcfc4d-fff3-443c-9f2d-7e5deba966e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:45.717366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcfc4d-fff3-443c-9f2d-7e5deba966e9 service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:45.717463 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7021f79e-57e7-44b5-8532-c74ee5c92b8a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:45.717668 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcfc4d-fff3-443c-9f2d-7e5deba966e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:45.717939 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcfc4d-fff3-443c-9f2d-7e5deba966e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:45.719827 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-4dfcfc4d-fff3-443c-9f2d-7e5deba966e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:45.789285 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcfc4d-fff3-443c-9f2d-7e5deba966e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:45.790765 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcfc4d-fff3-443c-9f2d-7e5deba966e9 service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:45.796217 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7021f79e-57e7-44b5-8532-c74ee5c92b8a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:45.800488 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 938/1886] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:45 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:56:45.800940 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e81aa-89d4-4062-a40f-390cb2d70a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:45.801657 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e81aa-89d4-4062-a40f-390cb2d70a74 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:45.801939 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e81aa-89d4-4062-a40f-390cb2d70a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:45.802249 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e81aa-89d4-4062-a40f-390cb2d70a74 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:45.802983 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e81aa-89d4-4062-a40f-390cb2d70a74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:45.804950 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e81aa-89d4-4062-a40f-390cb2d70a74 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:45.832243 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 949/1887] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:56:45 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:56:45.836612 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcfc4d-fff3-443c-9f2d-7e5deba966e9 service cinder] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:45.848262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcfc4d-fff3-443c-9f2d-7e5deba966e9 service cinder] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:45.930841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a6ae4b-aefe-411f-b589-121922b8bbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:45.931658 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6ae4b-aefe-411f-b589-121922b8bbb9 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:45.931942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6ae4b-aefe-411f-b589-121922b8bbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:45.932354 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6ae4b-aefe-411f-b589-121922b8bbb9 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:45.933192 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a6ae4b-aefe-411f-b589-121922b8bbb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:45.935282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a6ae4b-aefe-411f-b589-121922b8bbb9 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:45.944674 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:45.963058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 939/1888] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:56:45 2022] POST /identity/v3/users => generated 294 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:45.965243 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e81aa-89d4-4062-a40f-390cb2d70a74 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:45.978200 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e81aa-89d4-4062-a40f-390cb2d70a74 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:45.984611 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f9428-d1d9-4446-a69e-13935a1564d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:45.985405 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f9428-d1d9-4446-a69e-13935a1564d3 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:45.985882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f9428-d1d9-4446-a69e-13935a1564d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:45.986756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f9428-d1d9-4446-a69e-13935a1564d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:45.987071 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f9428-d1d9-4446-a69e-13935a1564d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:45.988768 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f9428-d1d9-4446-a69e-13935a1564d3 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:46.003474 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 950/1889] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:46.004603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a6ae4b-aefe-411f-b589-121922b8bbb9 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:46.015588 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a6ae4b-aefe-411f-b589-121922b8bbb9 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:46.021277 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9fb1f-89cb-4e31-bca1-3e4d56a5468b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:46.022122 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9fb1f-89cb-4e31-bca1-3e4d56a5468b admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:46.022531 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9fb1f-89cb-4e31-bca1-3e4d56a5468b admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:46.022870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9fb1f-89cb-4e31-bca1-3e4d56a5468b admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:46.023672 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9fb1f-89cb-4e31-bca1-3e4d56a5468b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:46.025414 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9fb1f-89cb-4e31-bca1-3e4d56a5468b admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:46.045658 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 940/1890] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:56:46.047622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f9428-d1d9-4446-a69e-13935a1564d3 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:46.071943 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f9428-d1d9-4446-a69e-13935a1564d3 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:46.079950 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337f234-ac2e-46f5-a4b8-d67a757dc567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:46.081136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337f234-ac2e-46f5-a4b8-d67a757dc567 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:46.081649 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337f234-ac2e-46f5-a4b8-d67a757dc567 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:46.082263 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337f234-ac2e-46f5-a4b8-d67a757dc567 admin admin] PATH_INFO: `/v3/projects/ddb7a79ba6b74f698b26fd50db4a5345/users/e4ccb99613214955ad41c9f02103957b/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:46.100668 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337f234-ac2e-46f5-a4b8-d67a757dc567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddb7a79ba6b74f698b26fd50db4a5345, user_id=e4ccb99613214955ad41c9f02103957b, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:46.103943 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337f234-ac2e-46f5-a4b8-d67a757dc567 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:46.122911 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 951/1891] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:46 2022] PUT /identity/v3/projects/ddb7a79ba6b74f698b26fd50db4a5345/users/e4ccb99613214955ad41c9f02103957b/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:46.134301 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e17e8-a709-483c-89f5-24993855421d None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:46.134609 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e17e8-a709-483c-89f5-24993855421d None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:46.134892 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e17e8-a709-483c-89f5-24993855421d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:46.169524 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-529e17e8-a709-483c-89f5-24993855421d None None] MFA Rules not processed for user `e4ccb99613214955ad41c9f02103957b`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:46.172465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-529e17e8-a709-483c-89f5-24993855421d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:46.245463 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 941/1892] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:46.250625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337f234-ac2e-46f5-a4b8-d67a757dc567 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:46.323714 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337f234-ac2e-46f5-a4b8-d67a757dc567 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:46.329277 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8e99d-df06-40c1-884a-f7e38953501e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:46.329950 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8e99d-df06-40c1-884a-f7e38953501e admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:46.330270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8e99d-df06-40c1-884a-f7e38953501e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:46.330624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8e99d-df06-40c1-884a-f7e38953501e admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:46.331275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8e99d-df06-40c1-884a-f7e38953501e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:46.333221 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8e99d-df06-40c1-884a-f7e38953501e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:46.366990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 952/1893] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:56:46 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:46.368119 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e17e8-a709-483c-89f5-24993855421d None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:46.379716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e17e8-a709-483c-89f5-24993855421d None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:46.465851 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6d487-a452-472b-b686-b5d46d8570c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:46.467244 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6d487-a452-472b-b686-b5d46d8570c1 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:46.468648 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6d487-a452-472b-b686-b5d46d8570c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:46.469215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6d487-a452-472b-b686-b5d46d8570c1 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:46.470517 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6d487-a452-472b-b686-b5d46d8570c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:46.473823 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6d487-a452-472b-b686-b5d46d8570c1 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:46.502249 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 942/1894] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:56:46 2022] POST /identity/v3/users => generated 299 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:56:46.506348 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8e99d-df06-40c1-884a-f7e38953501e admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:46.510352 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85756089-8875-4dd5-983c-5ebae615499f None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:46.510829 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85756089-8875-4dd5-983c-5ebae615499f None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:46.513876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85756089-8875-4dd5-983c-5ebae615499f None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:46.516618 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 943/1895] 10.222.0.78 () {54 vars in 921 bytes} [Wed May 25 14:56:46 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:56:46.541785 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069eecb-a663-4c84-b89f-8596288fd6ff None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:46.543802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069eecb-a663-4c84-b89f-8596288fd6ff None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:46.543802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069eecb-a663-4c84-b89f-8596288fd6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:46.548760 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8e99d-df06-40c1-884a-f7e38953501e admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:46.559982 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f829c6-5a25-4c18-afd9-3e957b553c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:46.560889 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f829c6-5a25-4c18-afd9-3e957b553c17 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:46.561304 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f829c6-5a25-4c18-afd9-3e957b553c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:46.561592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f829c6-5a25-4c18-afd9-3e957b553c17 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:46.563854 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f829c6-5a25-4c18-afd9-3e957b553c17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:46.565830 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f829c6-5a25-4c18-afd9-3e957b553c17 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:46.581375 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 953/1896] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:56:46.586686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f829c6-5a25-4c18-afd9-3e957b553c17 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:46.609372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:46.612074 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-d069eecb-a663-4c84-b89f-8596288fd6ff None None] MFA Rules not processed for user `e34e1d7ffbac49b0bde45829af0c6925`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:46.616549 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f829c6-5a25-4c18-afd9-3e957b553c17 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:46.617595 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d069eecb-a663-4c84-b89f-8596288fd6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:46.631488 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4c4af-cfe6-4b91-b67b-841adbdf0804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:46.633782 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4c4af-cfe6-4b91-b67b-841adbdf0804 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:46.634220 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4c4af-cfe6-4b91-b67b-841adbdf0804 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:46.635499 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4c4af-cfe6-4b91-b67b-841adbdf0804 admin admin] PATH_INFO: `/v3/projects/c325af4399c2479faf47b377b927a667/users/f46ecf12cefe4c168f2e660cd0068da5/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:46.664195 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4c4af-cfe6-4b91-b67b-841adbdf0804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c325af4399c2479faf47b377b927a667, user_id=f46ecf12cefe4c168f2e660cd0068da5, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:46.668226 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4c4af-cfe6-4b91-b67b-841adbdf0804 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:46.689233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 954/1897] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:46 2022] PUT /identity/v3/projects/c325af4399c2479faf47b377b927a667/users/f46ecf12cefe4c168f2e660cd0068da5/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:46.692918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4c4af-cfe6-4b91-b67b-841adbdf0804 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:46.708033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:46.732184 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 944/1898] 10.222.0.78 () {58 vars in 1018 bytes} [Wed May 25 14:56:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:56:46.744087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069eecb-a663-4c84-b89f-8596288fd6ff None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:46.745522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d069eecb-a663-4c84-b89f-8596288fd6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:46.759785 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069eecb-a663-4c84-b89f-8596288fd6ff None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:46.771077 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4c4af-cfe6-4b91-b67b-841adbdf0804 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:46.778811 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef11dc6-6859-4c96-9410-ded387a5bae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:46.779518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef11dc6-6859-4c96-9410-ded387a5bae1 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:46.779875 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef11dc6-6859-4c96-9410-ded387a5bae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:46.780898 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef11dc6-6859-4c96-9410-ded387a5bae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:46.781669 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef11dc6-6859-4c96-9410-ded387a5bae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:46.784022 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef11dc6-6859-4c96-9410-ded387a5bae1 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:46.797432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 955/1899] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:46.804502 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef11dc6-6859-4c96-9410-ded387a5bae1 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:46.821647 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef11dc6-6859-4c96-9410-ded387a5bae1 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:46.829104 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d82ad4-e69d-48b0-b884-e37b42b4f441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:46.830094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d82ad4-e69d-48b0-b884-e37b42b4f441 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:46.830377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d82ad4-e69d-48b0-b884-e37b42b4f441 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:46.830842 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d82ad4-e69d-48b0-b884-e37b42b4f441 admin admin] PATH_INFO: `/v3/projects/c325af4399c2479faf47b377b927a667/users/f46ecf12cefe4c168f2e660cd0068da5/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:46.833665 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d82ad4-e69d-48b0-b884-e37b42b4f441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c325af4399c2479faf47b377b927a667, user_id=f46ecf12cefe4c168f2e660cd0068da5, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:46.837409 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d82ad4-e69d-48b0-b884-e37b42b4f441 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:46.850664 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 956/1900] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:46 2022] PUT /identity/v3/projects/c325af4399c2479faf47b377b927a667/users/f46ecf12cefe4c168f2e660cd0068da5/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:46.861152 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf53b18-f88f-47ab-b44e-d3a841fc698b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:46.861442 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf53b18-f88f-47ab-b44e-d3a841fc698b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:46.861728 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf53b18-f88f-47ab-b44e-d3a841fc698b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:46.874112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345fbbe-a5fd-4652-9021-1ddde9372b22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:46.875281 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345fbbe-a5fd-4652-9021-1ddde9372b22 service nova] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:46.875779 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345fbbe-a5fd-4652-9021-1ddde9372b22 service nova] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:46.876281 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345fbbe-a5fd-4652-9021-1ddde9372b22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:46.893105 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0345fbbe-a5fd-4652-9021-1ddde9372b22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:46.899218 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-9cf53b18-f88f-47ab-b44e-d3a841fc698b None None] MFA Rules not processed for user `f46ecf12cefe4c168f2e660cd0068da5`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:46.900733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9cf53b18-f88f-47ab-b44e-d3a841fc698b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:46.906815 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0345fbbe-a5fd-4652-9021-1ddde9372b22 service nova] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:46.938388 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 945/1901] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:46.994838 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 957/1902] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:56:47.005985 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345fbbe-a5fd-4652-9021-1ddde9372b22 service nova] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:47.007422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-0345fbbe-a5fd-4652-9021-1ddde9372b22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:47.081131 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345fbbe-a5fd-4652-9021-1ddde9372b22 service nova] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:47.223854 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a02643-1ac3-4f14-a32d-4f813f0d8c77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:47.225620 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a02643-1ac3-4f14-a32d-4f813f0d8c77 service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:47.226141 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a02643-1ac3-4f14-a32d-4f813f0d8c77 service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:47.227341 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a02643-1ac3-4f14-a32d-4f813f0d8c77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:47.242170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a02643-1ac3-4f14-a32d-4f813f0d8c77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:47.244098 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a02643-1ac3-4f14-a32d-4f813f0d8c77 service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:47.259386 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 946/1903] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:47 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:48.743513 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf53b18-f88f-47ab-b44e-d3a841fc698b None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:48.756395 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf53b18-f88f-47ab-b44e-d3a841fc698b None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:48.761840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0d954-65dc-4332-85f1-250bde64d0aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:48.762673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0d954-65dc-4332-85f1-250bde64d0aa service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:48.762960 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0d954-65dc-4332-85f1-250bde64d0aa service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:48.763264 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0d954-65dc-4332-85f1-250bde64d0aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:48.826185 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:48.841828 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0d954-65dc-4332-85f1-250bde64d0aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:48.843917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0d954-65dc-4332-85f1-250bde64d0aa service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:48.857207 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 958/1904] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:49.715901 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a02643-1ac3-4f14-a32d-4f813f0d8c77 service cinder] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:49.719643 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-a4a02643-1ac3-4f14-a32d-4f813f0d8c77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:49.765074 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:49.802820 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a02643-1ac3-4f14-a32d-4f813f0d8c77 service cinder] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:49.808821 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86c15a-b892-4d24-b3c9-d4e1648248c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:49.809831 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86c15a-b892-4d24-b3c9-d4e1648248c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:49.810227 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86c15a-b892-4d24-b3c9-d4e1648248c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:49.810594 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86c15a-b892-4d24-b3c9-d4e1648248c9 admin admin] PATH_INFO: `/v3/users/070cde285fe04bc7bd5768fb803a0026` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:49.811976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86c15a-b892-4d24-b3c9-d4e1648248c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=070cde285fe04bc7bd5768fb803a0026)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:49.813900 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86c15a-b892-4d24-b3c9-d4e1648248c9 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:49.865274 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3d86c15a-b892-4d24-b3c9-d4e1648248c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '070cde285fe04bc7bd5768fb803a0026'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:49.878600 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3d86c15a-b892-4d24-b3c9-d4e1648248c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '070cde285fe04bc7bd5768fb803a0026'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:49.885819 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3d86c15a-b892-4d24-b3c9-d4e1648248c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '070cde285fe04bc7bd5768fb803a0026'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:49.886488 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3d86c15a-b892-4d24-b3c9-d4e1648248c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '070cde285fe04bc7bd5768fb803a0026'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:49.887219 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3d86c15a-b892-4d24-b3c9-d4e1648248c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '070cde285fe04bc7bd5768fb803a0026'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:49.903950 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 947/1905] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:56:49 2022] DELETE /identity/v3/users/070cde285fe04bc7bd5768fb803a0026 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:49.905632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef89b2-37da-4bc1-b989-bd015cdf2914 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:49.906090 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef89b2-37da-4bc1-b989-bd015cdf2914 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:49.906228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef89b2-37da-4bc1-b989-bd015cdf2914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:49.923706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-f3ef89b2-37da-4bc1-b989-bd015cdf2914 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:49.924957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f3ef89b2-37da-4bc1-b989-bd015cdf2914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:49.987905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 959/1906] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:56:49.998242 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86c15a-b892-4d24-b3c9-d4e1648248c9 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:49.999707 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-3d86c15a-b892-4d24-b3c9-d4e1648248c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:50.060078 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86c15a-b892-4d24-b3c9-d4e1648248c9 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:50.064807 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f48f3d-cdcb-478c-b90b-5659f6cae6c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:50.064807 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f48f3d-cdcb-478c-b90b-5659f6cae6c8 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:50.065141 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f48f3d-cdcb-478c-b90b-5659f6cae6c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:50.065299 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f48f3d-cdcb-478c-b90b-5659f6cae6c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:50.076425 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f48f3d-cdcb-478c-b90b-5659f6cae6c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:50.077873 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f48f3d-cdcb-478c-b90b-5659f6cae6c8 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:50.088442 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 948/1907] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:50.624841 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef89b2-37da-4bc1-b989-bd015cdf2914 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:50.627840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f3ef89b2-37da-4bc1-b989-bd015cdf2914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:50.658528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef89b2-37da-4bc1-b989-bd015cdf2914 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:50.675656 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe7372-9b3d-4f52-bb2a-cd2f669a3311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:50.683759 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe7372-9b3d-4f52-bb2a-cd2f669a3311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:50.687750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe7372-9b3d-4f52-bb2a-cd2f669a3311 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:50.688659 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe7372-9b3d-4f52-bb2a-cd2f669a3311 admin admin] PATH_INFO: `/v3/projects/fae42fd587ed4ee99b8d0c24a2ff20dd` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:50.691417 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abe7372-9b3d-4f52-bb2a-cd2f669a3311 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fae42fd587ed4ee99b8d0c24a2ff20dd)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:50.695412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abe7372-9b3d-4f52-bb2a-cd2f669a3311 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:50.853655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8abe7372-9b3d-4f52-bb2a-cd2f669a3311 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fae42fd587ed4ee99b8d0c24a2ff20dd', 'request_id': 'req-8abe7372-9b3d-4f52-bb2a-cd2f669a3311'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:50.874932 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8abe7372-9b3d-4f52-bb2a-cd2f669a3311 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fae42fd587ed4ee99b8d0c24a2ff20dd', 'request_id': 'req-8abe7372-9b3d-4f52-bb2a-cd2f669a3311'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:50.885906 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8abe7372-9b3d-4f52-bb2a-cd2f669a3311 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fae42fd587ed4ee99b8d0c24a2ff20dd', 'request_id': 'req-8abe7372-9b3d-4f52-bb2a-cd2f669a3311'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:50.895697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f48f3d-cdcb-478c-b90b-5659f6cae6c8 service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:50.903584 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d4f48f3d-cdcb-478c-b90b-5659f6cae6c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:50.937414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8abe7372-9b3d-4f52-bb2a-cd2f669a3311 admin admin] The token cache is being invalidate because project fae42fd587ed4ee99b8d0c24a2ff20dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:50.938780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8abe7372-9b3d-4f52-bb2a-cd2f669a3311 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:50.939846 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8abe7372-9b3d-4f52-bb2a-cd2f669a3311 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:50.945459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 960/1908] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:56:50 2022] DELETE /identity/v3/projects/fae42fd587ed4ee99b8d0c24a2ff20dd => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:50.946724 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe7372-9b3d-4f52-bb2a-cd2f669a3311 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:50.949863 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8abe7372-9b3d-4f52-bb2a-cd2f669a3311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:50.953228 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f48f3d-cdcb-478c-b90b-5659f6cae6c8 service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:51.069823 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:51.084609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe7372-9b3d-4f52-bb2a-cd2f669a3311 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:51.085875 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1468ea-8acb-4f89-970d-1c9ba29a6dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:51.087226 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1468ea-8acb-4f89-970d-1c9ba29a6dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:51.088431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1468ea-8acb-4f89-970d-1c9ba29a6dca admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:51.089998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1468ea-8acb-4f89-970d-1c9ba29a6dca admin admin] PATH_INFO: `/v3/users/4ff7b61a377749e0bc319b66097fd58b` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:51.091570 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceff483-74af-4ece-84ed-a0d6475b2459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:51.092676 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceff483-74af-4ece-84ed-a0d6475b2459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:51.093285 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceff483-74af-4ece-84ed-a0d6475b2459 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:51.093569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceff483-74af-4ece-84ed-a0d6475b2459 admin admin] PATH_INFO: `/v3/users/116802c8a90d4990ba762b3d39d29cb2` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:51.094707 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1468ea-8acb-4f89-970d-1c9ba29a6dca admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ff7b61a377749e0bc319b66097fd58b)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:51.094907 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceff483-74af-4ece-84ed-a0d6475b2459 admin admin] RBAC: Authorizing `identity:delete_user(user_id=116802c8a90d4990ba762b3d39d29cb2)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:51.097807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceff483-74af-4ece-84ed-a0d6475b2459 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:51.098327 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1468ea-8acb-4f89-970d-1c9ba29a6dca admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:51.121642 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:51.165385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ee1468ea-8acb-4f89-970d-1c9ba29a6dca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ff7b61a377749e0bc319b66097fd58b'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:51.174679 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3ceff483-74af-4ece-84ed-a0d6475b2459 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '116802c8a90d4990ba762b3d39d29cb2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:51.180546 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ee1468ea-8acb-4f89-970d-1c9ba29a6dca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ff7b61a377749e0bc319b66097fd58b'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:51.190670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ee1468ea-8acb-4f89-970d-1c9ba29a6dca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ff7b61a377749e0bc319b66097fd58b'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:51.191350 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ee1468ea-8acb-4f89-970d-1c9ba29a6dca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ff7b61a377749e0bc319b66097fd58b'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:51.192339 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ee1468ea-8acb-4f89-970d-1c9ba29a6dca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ff7b61a377749e0bc319b66097fd58b'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:51.193571 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3ceff483-74af-4ece-84ed-a0d6475b2459 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '116802c8a90d4990ba762b3d39d29cb2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:51.207154 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3ceff483-74af-4ece-84ed-a0d6475b2459 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '116802c8a90d4990ba762b3d39d29cb2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:51.207832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3ceff483-74af-4ece-84ed-a0d6475b2459 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '116802c8a90d4990ba762b3d39d29cb2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:51.208500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3ceff483-74af-4ece-84ed-a0d6475b2459 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '116802c8a90d4990ba762b3d39d29cb2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:51.209457 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 949/1909] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:56:50 2022] DELETE /identity/v3/users/4ff7b61a377749e0bc319b66097fd58b => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:51.214542 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56649e7e-def6-41b8-8d94-58bdd4777249 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:51.214542 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56649e7e-def6-41b8-8d94-58bdd4777249 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:51.214787 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56649e7e-def6-41b8-8d94-58bdd4777249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:51.228977 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 961/1910] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:56:50 2022] DELETE /identity/v3/users/116802c8a90d4990ba762b3d39d29cb2 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:51.270241 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-56649e7e-def6-41b8-8d94-58bdd4777249 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:51.271871 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-56649e7e-def6-41b8-8d94-58bdd4777249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:51.388381 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 950/1911] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:51.404870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceff483-74af-4ece-84ed-a0d6475b2459 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:51.407655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3ceff483-74af-4ece-84ed-a0d6475b2459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:51.532673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceff483-74af-4ece-84ed-a0d6475b2459 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:51.536936 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d573b-8d00-4bcc-a252-6c4612fc3da1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:51.537796 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d573b-8d00-4bcc-a252-6c4612fc3da1 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:51.538277 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d573b-8d00-4bcc-a252-6c4612fc3da1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:51.539004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d573b-8d00-4bcc-a252-6c4612fc3da1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:51.552720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d573b-8d00-4bcc-a252-6c4612fc3da1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:51.554434 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d573b-8d00-4bcc-a252-6c4612fc3da1 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:51.566435 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 962/1912] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:52.091400 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56649e7e-def6-41b8-8d94-58bdd4777249 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:52.102259 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-56649e7e-def6-41b8-8d94-58bdd4777249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:52.147432 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56649e7e-def6-41b8-8d94-58bdd4777249 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:52.183486 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754674d-88a9-41d4-bf86-77a76dfd4bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:52.190378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754674d-88a9-41d4-bf86-77a76dfd4bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:52.190702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754674d-88a9-41d4-bf86-77a76dfd4bea admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:52.190930 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754674d-88a9-41d4-bf86-77a76dfd4bea admin admin] PATH_INFO: `/v3/projects/59023bd2b85c40998f4e83c41b5bc462` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:52.195838 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754674d-88a9-41d4-bf86-77a76dfd4bea admin admin] RBAC: Authorizing `identity:delete_project(project_id=59023bd2b85c40998f4e83c41b5bc462)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:52.203656 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754674d-88a9-41d4-bf86-77a76dfd4bea admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:52.326193 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3754674d-88a9-41d4-bf86-77a76dfd4bea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59023bd2b85c40998f4e83c41b5bc462', 'request_id': 'req-3754674d-88a9-41d4-bf86-77a76dfd4bea'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:52.337506 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3754674d-88a9-41d4-bf86-77a76dfd4bea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59023bd2b85c40998f4e83c41b5bc462', 'request_id': 'req-3754674d-88a9-41d4-bf86-77a76dfd4bea'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:52.349028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3754674d-88a9-41d4-bf86-77a76dfd4bea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59023bd2b85c40998f4e83c41b5bc462', 'request_id': 'req-3754674d-88a9-41d4-bf86-77a76dfd4bea'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:52.365669 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:52.390875 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3754674d-88a9-41d4-bf86-77a76dfd4bea admin admin] The token cache is being invalidate because project 59023bd2b85c40998f4e83c41b5bc462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:52.392985 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3754674d-88a9-41d4-bf86-77a76dfd4bea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:52.396020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3754674d-88a9-41d4-bf86-77a76dfd4bea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:52.398691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 951/1913] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:56:52 2022] DELETE /identity/v3/projects/59023bd2b85c40998f4e83c41b5bc462 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:52.405755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a607ca-764b-4ce5-aafb-e6190367f822 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:52.406885 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a607ca-764b-4ce5-aafb-e6190367f822 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:52.406885 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a607ca-764b-4ce5-aafb-e6190367f822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:52.449273 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-c0a607ca-764b-4ce5-aafb-e6190367f822 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:52.453025 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c0a607ca-764b-4ce5-aafb-e6190367f822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:52.552728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754674d-88a9-41d4-bf86-77a76dfd4bea admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:52.554528 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-3754674d-88a9-41d4-bf86-77a76dfd4bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:52.627311 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 963/1914] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:52.631813 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5335899-f819-45d8-947c-9fa093a9a41c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:52.631970 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5335899-f819-45d8-947c-9fa093a9a41c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:52.632864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5335899-f819-45d8-947c-9fa093a9a41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:52.658435 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-e5335899-f819-45d8-947c-9fa093a9a41c None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:52.661312 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e5335899-f819-45d8-947c-9fa093a9a41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:52.672852 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 964/1915] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:56:52.678102 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5335899-f819-45d8-947c-9fa093a9a41c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:52.694305 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5335899-f819-45d8-947c-9fa093a9a41c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:52.700580 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754674d-88a9-41d4-bf86-77a76dfd4bea admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:52.700768 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f6169-0feb-4b6f-8738-925406db0038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:52.701404 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f6169-0feb-4b6f-8738-925406db0038 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:52.701744 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f6169-0feb-4b6f-8738-925406db0038 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:52.702039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f6169-0feb-4b6f-8738-925406db0038 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:52.704538 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f6169-0feb-4b6f-8738-925406db0038 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:56:52.704864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f6169-0feb-4b6f-8738-925406db0038 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:52.707129 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f6169-0feb-4b6f-8738-925406db0038 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:52.716966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b798c-12dd-4402-b4b6-9f82a88a6314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:52.717873 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b798c-12dd-4402-b4b6-9f82a88a6314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:52.718191 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b798c-12dd-4402-b4b6-9f82a88a6314 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:52.718467 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b798c-12dd-4402-b4b6-9f82a88a6314 admin admin] PATH_INFO: `/v3/projects/5aacf2979fb34111a51fbaf0f0034b1c` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:52.721907 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b798c-12dd-4402-b4b6-9f82a88a6314 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5aacf2979fb34111a51fbaf0f0034b1c)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:52.723924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b798c-12dd-4402-b4b6-9f82a88a6314 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:52.737191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 965/1916] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:56:52.739796 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f6169-0feb-4b6f-8738-925406db0038 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:52.787348 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f6169-0feb-4b6f-8738-925406db0038 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:52.809399 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb158601-3421-4395-a1e6-ed0e84457d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:52.810861 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb158601-3421-4395-a1e6-ed0e84457d6f admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:52.811375 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb158601-3421-4395-a1e6-ed0e84457d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:52.811684 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb158601-3421-4395-a1e6-ed0e84457d6f admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:52.812390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb158601-3421-4395-a1e6-ed0e84457d6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:52.814798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb158601-3421-4395-a1e6-ed0e84457d6f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:52.817655 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b33b798c-12dd-4402-b4b6-9f82a88a6314 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5aacf2979fb34111a51fbaf0f0034b1c', 'request_id': 'req-b33b798c-12dd-4402-b4b6-9f82a88a6314'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:52.824272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b33b798c-12dd-4402-b4b6-9f82a88a6314 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5aacf2979fb34111a51fbaf0f0034b1c', 'request_id': 'req-b33b798c-12dd-4402-b4b6-9f82a88a6314'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:52.826994 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:52.831998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b33b798c-12dd-4402-b4b6-9f82a88a6314 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5aacf2979fb34111a51fbaf0f0034b1c', 'request_id': 'req-b33b798c-12dd-4402-b4b6-9f82a88a6314'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:52.852409 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b33b798c-12dd-4402-b4b6-9f82a88a6314 admin admin] The token cache is being invalidate because project 5aacf2979fb34111a51fbaf0f0034b1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:52.853396 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 966/1917] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:56:52 2022] POST /identity/v3/projects => generated 365 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:52.853684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b33b798c-12dd-4402-b4b6-9f82a88a6314 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:52.855657 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b33b798c-12dd-4402-b4b6-9f82a88a6314 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:52.857307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb158601-3421-4395-a1e6-ed0e84457d6f admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:52.859171 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 952/1918] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:56:52 2022] DELETE /identity/v3/projects/5aacf2979fb34111a51fbaf0f0034b1c => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:52.863282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b798c-12dd-4402-b4b6-9f82a88a6314 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:52.864054 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-fb158601-3421-4395-a1e6-ed0e84457d6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:52.864700 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-b33b798c-12dd-4402-b4b6-9f82a88a6314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:52.950635 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b798c-12dd-4402-b4b6-9f82a88a6314 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:52.960243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30574a42-b3cd-48c3-9a2b-3b60ff3b1651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:52.961143 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30574a42-b3cd-48c3-9a2b-3b60ff3b1651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:52.961256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb158601-3421-4395-a1e6-ed0e84457d6f admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:52.961450 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30574a42-b3cd-48c3-9a2b-3b60ff3b1651 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:52.961682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30574a42-b3cd-48c3-9a2b-3b60ff3b1651 admin admin] PATH_INFO: `/v3/users/9ee7927c702b456e9a3ec1a286c8f6e4` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:52.963547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30574a42-b3cd-48c3-9a2b-3b60ff3b1651 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ee7927c702b456e9a3ec1a286c8f6e4)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:52.966100 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30574a42-b3cd-48c3-9a2b-3b60ff3b1651 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:52.966812 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d207529-be40-4eaa-b230-edb990dfd1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:52.967700 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d207529-be40-4eaa-b230-edb990dfd1a5 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:52.967988 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d207529-be40-4eaa-b230-edb990dfd1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:52.968269 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d207529-be40-4eaa-b230-edb990dfd1a5 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:52.969117 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d207529-be40-4eaa-b230-edb990dfd1a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:52.971126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d207529-be40-4eaa-b230-edb990dfd1a5 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:52.996126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 967/1919] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:56:52 2022] POST /identity/v3/users => generated 283 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:56:53.001104 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d207529-be40-4eaa-b230-edb990dfd1a5 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:53.020688 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:53.021801 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d207529-be40-4eaa-b230-edb990dfd1a5 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:53.027988 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d5249-e13d-4a53-b1e3-edbf46d0e2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:53.028718 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d5249-e13d-4a53-b1e3-edbf46d0e2ab admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:53.029025 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d5249-e13d-4a53-b1e3-edbf46d0e2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:53.029456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d5249-e13d-4a53-b1e3-edbf46d0e2ab admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:53.031007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d5249-e13d-4a53-b1e3-edbf46d0e2ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:53.033007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d5249-e13d-4a53-b1e3-edbf46d0e2ab admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:53.048282 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 968/1920] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:53.052258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-30574a42-b3cd-48c3-9a2b-3b60ff3b1651 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ee7927c702b456e9a3ec1a286c8f6e4'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:53.055446 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d5249-e13d-4a53-b1e3-edbf46d0e2ab admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:53.070280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-30574a42-b3cd-48c3-9a2b-3b60ff3b1651 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ee7927c702b456e9a3ec1a286c8f6e4'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:53.076616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d5249-e13d-4a53-b1e3-edbf46d0e2ab admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:53.081512 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-30574a42-b3cd-48c3-9a2b-3b60ff3b1651 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ee7927c702b456e9a3ec1a286c8f6e4'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:53.082922 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-30574a42-b3cd-48c3-9a2b-3b60ff3b1651 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ee7927c702b456e9a3ec1a286c8f6e4'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:53.083037 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef8792-fc71-467b-86c2-c0d31287adc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:53.083732 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef8792-fc71-467b-86c2-c0d31287adc8 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:53.084051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef8792-fc71-467b-86c2-c0d31287adc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:53.084415 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef8792-fc71-467b-86c2-c0d31287adc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:53.085266 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef8792-fc71-467b-86c2-c0d31287adc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:53.085375 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-30574a42-b3cd-48c3-9a2b-3b60ff3b1651 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ee7927c702b456e9a3ec1a286c8f6e4'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:53.087053 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef8792-fc71-467b-86c2-c0d31287adc8 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:53.100910 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 953/1921] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:56:52 2022] DELETE /identity/v3/users/9ee7927c702b456e9a3ec1a286c8f6e4 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:56:53.101335 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 969/1922] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:53.106787 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30574a42-b3cd-48c3-9a2b-3b60ff3b1651 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:53.108550 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-30574a42-b3cd-48c3-9a2b-3b60ff3b1651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:53.220215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30574a42-b3cd-48c3-9a2b-3b60ff3b1651 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:53.225816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f14e3-28a4-4f52-ad68-0d39405d2a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:53.226758 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f14e3-28a4-4f52-ad68-0d39405d2a9d admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:53.227043 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f14e3-28a4-4f52-ad68-0d39405d2a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:53.227321 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f14e3-28a4-4f52-ad68-0d39405d2a9d admin admin] PATH_INFO: `/v3/projects/aaaa40cb97c3403f88d72657c5b66215/users/d1a8dd77d27f4756a438e675b3e98705/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:53.260306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f14e3-28a4-4f52-ad68-0d39405d2a9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaaa40cb97c3403f88d72657c5b66215, user_id=d1a8dd77d27f4756a438e675b3e98705, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:53.264095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f14e3-28a4-4f52-ad68-0d39405d2a9d admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:53.299424 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 954/1923] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:53 2022] PUT /identity/v3/projects/aaaa40cb97c3403f88d72657c5b66215/users/d1a8dd77d27f4756a438e675b3e98705/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:53.309883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4a256-cc3d-4683-a2bf-2c2a056e8902 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:53.310426 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4a256-cc3d-4683-a2bf-2c2a056e8902 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:53.311501 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4a256-cc3d-4683-a2bf-2c2a056e8902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:53.349667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-a7e4a256-cc3d-4683-a2bf-2c2a056e8902 None None] MFA Rules not processed for user `d1a8dd77d27f4756a438e675b3e98705`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:53.350725 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a7e4a256-cc3d-4683-a2bf-2c2a056e8902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:53.501889 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 970/1924] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:53 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:53.510633 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f14e3-28a4-4f52-ad68-0d39405d2a9d admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:53.514094 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-bc7f14e3-28a4-4f52-ad68-0d39405d2a9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:53.624628 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f14e3-28a4-4f52-ad68-0d39405d2a9d admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:53.638073 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076c588-89d4-441a-9464-a645c267348e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:53.638340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076c588-89d4-441a-9464-a645c267348e service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:53.638563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076c588-89d4-441a-9464-a645c267348e service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:53.638848 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076c588-89d4-441a-9464-a645c267348e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:53.656425 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076c588-89d4-441a-9464-a645c267348e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:53.657948 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076c588-89d4-441a-9464-a645c267348e service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:53.681818 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 955/1925] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:53 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:56:53.751709 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4a256-cc3d-4683-a2bf-2c2a056e8902 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:53.753614 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a7e4a256-cc3d-4683-a2bf-2c2a056e8902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:53.775911 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4a256-cc3d-4683-a2bf-2c2a056e8902 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:53.783109 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fccee-5f2e-4f8b-a7f0-d9c1e7f4b83f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:53.784181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fccee-5f2e-4f8b-a7f0-d9c1e7f4b83f service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:53.784661 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fccee-5f2e-4f8b-a7f0-d9c1e7f4b83f service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:53.785222 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fccee-5f2e-4f8b-a7f0-d9c1e7f4b83f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:53.788072 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-bc2fccee-5f2e-4f8b-a7f0-d9c1e7f4b83f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:53.891441 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:53.913304 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2fccee-5f2e-4f8b-a7f0-d9c1e7f4b83f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:53.914985 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2fccee-5f2e-4f8b-a7f0-d9c1e7f4b83f service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:53.918299 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076c588-89d4-441a-9464-a645c267348e service cinder] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:53.919809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-5076c588-89d4-441a-9464-a645c267348e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:53.928974 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 971/1926] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:54.002855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:54.089102 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076c588-89d4-441a-9464-a645c267348e service cinder] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:54.108305 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0813598-7e6b-4f4d-8424-6d0c45ea8a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:54.109330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0813598-7e6b-4f4d-8424-6d0c45ea8a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:54.109713 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0813598-7e6b-4f4d-8424-6d0c45ea8a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:54.110088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0813598-7e6b-4f4d-8424-6d0c45ea8a8f admin admin] PATH_INFO: `/v3/projects/2e187010915e4ea6a492217b229c8916` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:54.113518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0813598-7e6b-4f4d-8424-6d0c45ea8a8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e187010915e4ea6a492217b229c8916)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:54.114869 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0813598-7e6b-4f4d-8424-6d0c45ea8a8f admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:54.227107 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b0813598-7e6b-4f4d-8424-6d0c45ea8a8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e187010915e4ea6a492217b229c8916', 'request_id': 'req-b0813598-7e6b-4f4d-8424-6d0c45ea8a8f'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:54.235923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b0813598-7e6b-4f4d-8424-6d0c45ea8a8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e187010915e4ea6a492217b229c8916', 'request_id': 'req-b0813598-7e6b-4f4d-8424-6d0c45ea8a8f'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:54.245281 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b0813598-7e6b-4f4d-8424-6d0c45ea8a8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e187010915e4ea6a492217b229c8916', 'request_id': 'req-b0813598-7e6b-4f4d-8424-6d0c45ea8a8f'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:54.271975 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b0813598-7e6b-4f4d-8424-6d0c45ea8a8f admin admin] The token cache is being invalidate because project 2e187010915e4ea6a492217b229c8916 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:54.272477 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b0813598-7e6b-4f4d-8424-6d0c45ea8a8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:54.274042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b0813598-7e6b-4f4d-8424-6d0c45ea8a8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:54.279062 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 956/1927] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:56:53 2022] DELETE /identity/v3/projects/2e187010915e4ea6a492217b229c8916 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:56:54.286128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d355a-088e-45cf-9c28-b8c6601a0b10 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:54.286417 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d355a-088e-45cf-9c28-b8c6601a0b10 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:54.286699 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d355a-088e-45cf-9c28-b8c6601a0b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:54.319081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-392d355a-088e-45cf-9c28-b8c6601a0b10 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:54.320840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-392d355a-088e-45cf-9c28-b8c6601a0b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:54.404218 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 972/1928] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:56:54.412115 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67137f73-df03-47bf-a578-974ee93270dd None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:54.412666 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67137f73-df03-47bf-a578-974ee93270dd None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:54.412942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67137f73-df03-47bf-a578-974ee93270dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:54.436669 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-67137f73-df03-47bf-a578-974ee93270dd None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:54.437897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-67137f73-df03-47bf-a578-974ee93270dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:54.447189 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 957/1929] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:54.455133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d355a-088e-45cf-9c28-b8c6601a0b10 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:54.466004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d355a-088e-45cf-9c28-b8c6601a0b10 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:54.472126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1634cd8-6e2d-439f-9996-8f232047d869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:54.472811 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1634cd8-6e2d-439f-9996-8f232047d869 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:54.473090 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1634cd8-6e2d-439f-9996-8f232047d869 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:54.473367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1634cd8-6e2d-439f-9996-8f232047d869 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:54.474137 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1634cd8-6e2d-439f-9996-8f232047d869 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:56:54.474456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1634cd8-6e2d-439f-9996-8f232047d869 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:54.476071 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1634cd8-6e2d-439f-9996-8f232047d869 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:54.489835 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 973/1930] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:56:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:56:54.497594 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67137f73-df03-47bf-a578-974ee93270dd None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:54.508565 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67137f73-df03-47bf-a578-974ee93270dd None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:54.516396 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a31d0-daef-4885-a536-f07253608620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:54.517231 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a31d0-daef-4885-a536-f07253608620 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:54.517590 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a31d0-daef-4885-a536-f07253608620 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:54.517868 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a31d0-daef-4885-a536-f07253608620 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:54.518631 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a31d0-daef-4885-a536-f07253608620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:54.520816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a31d0-daef-4885-a536-f07253608620 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:54.562813 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 958/1931] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:56:54 2022] POST /identity/v3/projects => generated 383 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:54.567140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1634cd8-6e2d-439f-9996-8f232047d869 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:54.580143 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1634cd8-6e2d-439f-9996-8f232047d869 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:54.655596 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de34b7d-411c-4562-b5b1-a5d16325076f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:54.656280 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de34b7d-411c-4562-b5b1-a5d16325076f admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:54.656621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de34b7d-411c-4562-b5b1-a5d16325076f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:54.656890 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de34b7d-411c-4562-b5b1-a5d16325076f admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:54.657634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de34b7d-411c-4562-b5b1-a5d16325076f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:54.659755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de34b7d-411c-4562-b5b1-a5d16325076f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:54.691771 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 974/1932] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:56:54 2022] POST /identity/v3/users => generated 292 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:54.693007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a31d0-daef-4885-a536-f07253608620 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:54.705841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a31d0-daef-4885-a536-f07253608620 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:54.719975 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4aceec-49ba-44bb-94de-c41311438892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:54.720795 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4aceec-49ba-44bb-94de-c41311438892 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:54.721135 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4aceec-49ba-44bb-94de-c41311438892 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:54.721662 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4aceec-49ba-44bb-94de-c41311438892 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:54.722448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4aceec-49ba-44bb-94de-c41311438892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:54.724779 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4aceec-49ba-44bb-94de-c41311438892 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:54.742122 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 959/1933] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:54.744130 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de34b7d-411c-4562-b5b1-a5d16325076f admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:54.762736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de34b7d-411c-4562-b5b1-a5d16325076f admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:54.769611 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b96da7-eb3b-42d8-b96b-d29c362ff203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:54.770779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b96da7-eb3b-42d8-b96b-d29c362ff203 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:54.771211 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b96da7-eb3b-42d8-b96b-d29c362ff203 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:54.771684 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b96da7-eb3b-42d8-b96b-d29c362ff203 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:54.772698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b96da7-eb3b-42d8-b96b-d29c362ff203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:54.775270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b96da7-eb3b-42d8-b96b-d29c362ff203 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:54.792618 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 975/1934] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:54.797288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4aceec-49ba-44bb-94de-c41311438892 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:54.810593 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4aceec-49ba-44bb-94de-c41311438892 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:54.821580 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887d13d-06b9-4d09-a42f-99134838f27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:54.825074 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887d13d-06b9-4d09-a42f-99134838f27e admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:54.825074 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887d13d-06b9-4d09-a42f-99134838f27e admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:54.825074 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887d13d-06b9-4d09-a42f-99134838f27e admin admin] PATH_INFO: `/v3/projects/0259cabe6366480b874562418bd7ee32/users/e046ac3cf4da44cdaead991b6060f1f4/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:54.850201 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e887d13d-06b9-4d09-a42f-99134838f27e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0259cabe6366480b874562418bd7ee32, user_id=e046ac3cf4da44cdaead991b6060f1f4, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:54.854245 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e887d13d-06b9-4d09-a42f-99134838f27e admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:54.892080 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 960/1935] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:54 2022] PUT /identity/v3/projects/0259cabe6366480b874562418bd7ee32/users/e046ac3cf4da44cdaead991b6060f1f4/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:54.894889 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3361fac-b85f-4d91-8006-7c20558cc549 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:54.895405 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3361fac-b85f-4d91-8006-7c20558cc549 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:54.895832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3361fac-b85f-4d91-8006-7c20558cc549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:54.935367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-c3361fac-b85f-4d91-8006-7c20558cc549 None None] MFA Rules not processed for user `e046ac3cf4da44cdaead991b6060f1f4`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:54.937356 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c3361fac-b85f-4d91-8006-7c20558cc549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:55.005458 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:55.054276 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 976/1936] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:56:55.059268 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887d13d-06b9-4d09-a42f-99134838f27e admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:55.216214 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887d13d-06b9-4d09-a42f-99134838f27e admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:55.222893 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e638fb4-e7ab-4554-a2a3-57d23b891aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:55.223775 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e638fb4-e7ab-4554-a2a3-57d23b891aa3 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:55.224275 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e638fb4-e7ab-4554-a2a3-57d23b891aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:55.224696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e638fb4-e7ab-4554-a2a3-57d23b891aa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:55.225569 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e638fb4-e7ab-4554-a2a3-57d23b891aa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:55.228200 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e638fb4-e7ab-4554-a2a3-57d23b891aa3 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:55.234216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:55.271292 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 961/1937] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:56:55 2022] POST /identity/v3/projects => generated 379 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:55.275813 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3361fac-b85f-4d91-8006-7c20558cc549 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:55.290620 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3361fac-b85f-4d91-8006-7c20558cc549 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:55.387551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f8d2c-f37d-4f4b-998c-20f68ea58d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:55.388796 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f8d2c-f37d-4f4b-998c-20f68ea58d47 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:55.389286 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f8d2c-f37d-4f4b-998c-20f68ea58d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:55.390381 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f8d2c-f37d-4f4b-998c-20f68ea58d47 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:55.391319 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f8d2c-f37d-4f4b-998c-20f68ea58d47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:55.393514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f8d2c-f37d-4f4b-998c-20f68ea58d47 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:55.419865 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 977/1938] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:56:55 2022] POST /identity/v3/users => generated 296 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:56:55.425325 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e638fb4-e7ab-4554-a2a3-57d23b891aa3 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:55.443252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e638fb4-e7ab-4554-a2a3-57d23b891aa3 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:55.451665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0401136-4a09-42e8-aa1b-bdfebea8bc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:55.452773 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0401136-4a09-42e8-aa1b-bdfebea8bc0f admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:55.453231 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0401136-4a09-42e8-aa1b-bdfebea8bc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:55.453691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0401136-4a09-42e8-aa1b-bdfebea8bc0f admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:55.455234 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0401136-4a09-42e8-aa1b-bdfebea8bc0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:55.458640 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0401136-4a09-42e8-aa1b-bdfebea8bc0f admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:55.480955 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 962/1939] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:56:55.485024 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f8d2c-f37d-4f4b-998c-20f68ea58d47 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:55.502206 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f8d2c-f37d-4f4b-998c-20f68ea58d47 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:55.508962 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ac3be-26c5-4cec-a12d-e704476263b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:55.510199 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ac3be-26c5-4cec-a12d-e704476263b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:55.510487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ac3be-26c5-4cec-a12d-e704476263b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:55.510768 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ac3be-26c5-4cec-a12d-e704476263b8 admin admin] PATH_INFO: `/v3/projects/6b6dc6bd6ec04d30875c69c354ccc219/users/2ee3e13df6d443ce97139c5753885877/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:55.531461 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37ac3be-26c5-4cec-a12d-e704476263b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b6dc6bd6ec04d30875c69c354ccc219, user_id=2ee3e13df6d443ce97139c5753885877, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:55.534583 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37ac3be-26c5-4cec-a12d-e704476263b8 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:55.550865 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 978/1940] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:55 2022] PUT /identity/v3/projects/6b6dc6bd6ec04d30875c69c354ccc219/users/2ee3e13df6d443ce97139c5753885877/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:55.554782 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0401136-4a09-42e8-aa1b-bdfebea8bc0f admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:55.566232 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0401136-4a09-42e8-aa1b-bdfebea8bc0f admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:55.652326 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb5f78-a048-433f-88a3-33e7a8f2446e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:55.652326 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb5f78-a048-433f-88a3-33e7a8f2446e admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:55.652564 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb5f78-a048-433f-88a3-33e7a8f2446e admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:55.652759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb5f78-a048-433f-88a3-33e7a8f2446e admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:55.653603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb5f78-a048-433f-88a3-33e7a8f2446e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:55.656013 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb5f78-a048-433f-88a3-33e7a8f2446e admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:55.682372 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 963/1941] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:56:55.685449 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ac3be-26c5-4cec-a12d-e704476263b8 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:55.699392 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ac3be-26c5-4cec-a12d-e704476263b8 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:55.707157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b95902-d39b-476c-a4e2-5cbd9b2c1c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:55.710303 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b95902-d39b-476c-a4e2-5cbd9b2c1c77 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:55.710733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b95902-d39b-476c-a4e2-5cbd9b2c1c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:55.711106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b95902-d39b-476c-a4e2-5cbd9b2c1c77 admin admin] PATH_INFO: `/v3/projects/6b6dc6bd6ec04d30875c69c354ccc219/users/2ee3e13df6d443ce97139c5753885877/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:55.713771 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b95902-d39b-476c-a4e2-5cbd9b2c1c77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b6dc6bd6ec04d30875c69c354ccc219, user_id=2ee3e13df6d443ce97139c5753885877, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:55.717498 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b95902-d39b-476c-a4e2-5cbd9b2c1c77 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:55.739841 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 979/1942] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:56:55 2022] PUT /identity/v3/projects/6b6dc6bd6ec04d30875c69c354ccc219/users/2ee3e13df6d443ce97139c5753885877/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:55.751454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6540d-9b4d-4119-9dfa-825e07c969de None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:55.751776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6540d-9b4d-4119-9dfa-825e07c969de None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:55.752084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6540d-9b4d-4119-9dfa-825e07c969de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:55.783007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-69c6540d-9b4d-4119-9dfa-825e07c969de None None] MFA Rules not processed for user `2ee3e13df6d443ce97139c5753885877`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:55.784298 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-69c6540d-9b4d-4119-9dfa-825e07c969de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:55.889992 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 964/1943] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:55 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:56:55.904365 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b95902-d39b-476c-a4e2-5cbd9b2c1c77 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:55.906728 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e8b95902-d39b-476c-a4e2-5cbd9b2c1c77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:55.972399 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b95902-d39b-476c-a4e2-5cbd9b2c1c77 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:55.976231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c4fe2-13ca-491f-adc4-34523c830be1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:55.976972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c4fe2-13ca-491f-adc4-34523c830be1 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:55.977325 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c4fe2-13ca-491f-adc4-34523c830be1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:55.977672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c4fe2-13ca-491f-adc4-34523c830be1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:55.987780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c4fe2-13ca-491f-adc4-34523c830be1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:55.989223 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c4fe2-13ca-491f-adc4-34523c830be1 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:56.000115 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 980/1944] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:55 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:56.114641 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6540d-9b4d-4119-9dfa-825e07c969de None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:56.127233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6540d-9b4d-4119-9dfa-825e07c969de None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:56.131605 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1436e1c9-e681-4266-8b7d-c2f8902cec09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:56.132252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436e1c9-e681-4266-8b7d-c2f8902cec09 service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:56.132596 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436e1c9-e681-4266-8b7d-c2f8902cec09 service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:56.132923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436e1c9-e681-4266-8b7d-c2f8902cec09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:56.191215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:56.194994 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1436e1c9-e681-4266-8b7d-c2f8902cec09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:56.196893 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1436e1c9-e681-4266-8b7d-c2f8902cec09 service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:56.209983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 965/1945] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:57.039777 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c4fe2-13ca-491f-adc4-34523c830be1 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:57.041473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-741c4fe2-13ca-491f-adc4-34523c830be1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:57.097513 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:57.126534 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c4fe2-13ca-491f-adc4-34523c830be1 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:57.131568 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d93f9-3bc9-40a8-a8d1-33b311d39f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:57.132433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d93f9-3bc9-40a8-a8d1-33b311d39f21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:57.132713 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d93f9-3bc9-40a8-a8d1-33b311d39f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:57.132987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d93f9-3bc9-40a8-a8d1-33b311d39f21 admin admin] PATH_INFO: `/v3/users/c40946dc499e497aa717c8eb7278b30d` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:57.134247 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3d93f9-3bc9-40a8-a8d1-33b311d39f21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c40946dc499e497aa717c8eb7278b30d)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:57.136230 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3d93f9-3bc9-40a8-a8d1-33b311d39f21 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:57.188025 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-7f3d93f9-3bc9-40a8-a8d1-33b311d39f21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c40946dc499e497aa717c8eb7278b30d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:57.200547 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-7f3d93f9-3bc9-40a8-a8d1-33b311d39f21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c40946dc499e497aa717c8eb7278b30d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:57.209670 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-7f3d93f9-3bc9-40a8-a8d1-33b311d39f21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c40946dc499e497aa717c8eb7278b30d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:57.210349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-7f3d93f9-3bc9-40a8-a8d1-33b311d39f21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c40946dc499e497aa717c8eb7278b30d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:57.211064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-7f3d93f9-3bc9-40a8-a8d1-33b311d39f21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c40946dc499e497aa717c8eb7278b30d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:57.224319 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 981/1946] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:56:57 2022] DELETE /identity/v3/users/c40946dc499e497aa717c8eb7278b30d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:57.228997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a8f57-54ad-452e-848f-91de55b70948 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:57.230849 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a8f57-54ad-452e-848f-91de55b70948 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:57.231200 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a8f57-54ad-452e-848f-91de55b70948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:57.251434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-e82a8f57-54ad-452e-848f-91de55b70948 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:56:57.253056 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-e82a8f57-54ad-452e-848f-91de55b70948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:57.334735 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 966/1947] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:56:57.344439 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d93f9-3bc9-40a8-a8d1-33b311d39f21 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:57.345843 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7f3d93f9-3bc9-40a8-a8d1-33b311d39f21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:57.428958 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d93f9-3bc9-40a8-a8d1-33b311d39f21 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:57.435530 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf81b5f-b91d-442c-822a-cc130c95bfee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:57.436967 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf81b5f-b91d-442c-822a-cc130c95bfee service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:57.437489 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf81b5f-b91d-442c-822a-cc130c95bfee service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:57.438112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf81b5f-b91d-442c-822a-cc130c95bfee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:57.448467 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf81b5f-b91d-442c-822a-cc130c95bfee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:57.450510 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf81b5f-b91d-442c-822a-cc130c95bfee service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:57.460597 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 982/1948] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:56:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:57.809184 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a8f57-54ad-452e-848f-91de55b70948 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:57.811909 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-e82a8f57-54ad-452e-848f-91de55b70948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:57.833867 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a8f57-54ad-452e-848f-91de55b70948 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:57.842291 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5800b03-42a1-4ef8-ab9a-39a3110af62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:57.843156 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5800b03-42a1-4ef8-ab9a-39a3110af62c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:57.844234 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5800b03-42a1-4ef8-ab9a-39a3110af62c admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:57.844841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5800b03-42a1-4ef8-ab9a-39a3110af62c admin admin] PATH_INFO: `/v3/projects/f2a776b2cf004d6eb1b77d6801208259` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:57.848748 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5800b03-42a1-4ef8-ab9a-39a3110af62c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2a776b2cf004d6eb1b77d6801208259)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:57.851989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5800b03-42a1-4ef8-ab9a-39a3110af62c admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:57.926227 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f5800b03-42a1-4ef8-ab9a-39a3110af62c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2a776b2cf004d6eb1b77d6801208259', 'request_id': 'req-f5800b03-42a1-4ef8-ab9a-39a3110af62c'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:57.936703 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f5800b03-42a1-4ef8-ab9a-39a3110af62c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2a776b2cf004d6eb1b77d6801208259', 'request_id': 'req-f5800b03-42a1-4ef8-ab9a-39a3110af62c'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:57.945391 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f5800b03-42a1-4ef8-ab9a-39a3110af62c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2a776b2cf004d6eb1b77d6801208259', 'request_id': 'req-f5800b03-42a1-4ef8-ab9a-39a3110af62c'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:57.972839 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f5800b03-42a1-4ef8-ab9a-39a3110af62c admin admin] The token cache is being invalidate because project f2a776b2cf004d6eb1b77d6801208259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:57.973043 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f5800b03-42a1-4ef8-ab9a-39a3110af62c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:57.974720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f5800b03-42a1-4ef8-ab9a-39a3110af62c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:57.980864 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 967/1949] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:56:57 2022] DELETE /identity/v3/projects/f2a776b2cf004d6eb1b77d6801208259 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:57.982683 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf81b5f-b91d-442c-822a-cc130c95bfee service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:58.019042 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf81b5f-b91d-442c-822a-cc130c95bfee service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:58.087764 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9571b5-7dd9-47c2-9cab-8d99de66650b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:58.089459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9571b5-7dd9-47c2-9cab-8d99de66650b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:58.089771 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9571b5-7dd9-47c2-9cab-8d99de66650b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:58.090249 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9571b5-7dd9-47c2-9cab-8d99de66650b admin admin] PATH_INFO: `/v3/users/e4ccb99613214955ad41c9f02103957b` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:58.091904 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9571b5-7dd9-47c2-9cab-8d99de66650b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4ccb99613214955ad41c9f02103957b)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:58.095985 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9571b5-7dd9-47c2-9cab-8d99de66650b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:58.100025 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:58.180600 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4e9571b5-7dd9-47c2-9cab-8d99de66650b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4ccb99613214955ad41c9f02103957b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:58.200289 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4e9571b5-7dd9-47c2-9cab-8d99de66650b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4ccb99613214955ad41c9f02103957b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:58.209178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4e9571b5-7dd9-47c2-9cab-8d99de66650b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4ccb99613214955ad41c9f02103957b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:58.209940 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4e9571b5-7dd9-47c2-9cab-8d99de66650b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4ccb99613214955ad41c9f02103957b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:58.210610 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4e9571b5-7dd9-47c2-9cab-8d99de66650b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4ccb99613214955ad41c9f02103957b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:58.227359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 983/1950] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:56:57 2022] DELETE /identity/v3/users/e4ccb99613214955ad41c9f02103957b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:58.567722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5800b03-42a1-4ef8-ab9a-39a3110af62c admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:58.569248 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f5800b03-42a1-4ef8-ab9a-39a3110af62c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:58.601687 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5800b03-42a1-4ef8-ab9a-39a3110af62c admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:58.708041 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43fe91f-827d-4a0a-941d-e327fad62be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:58.709319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fe91f-827d-4a0a-941d-e327fad62be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:58.709894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fe91f-827d-4a0a-941d-e327fad62be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:58.710269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fe91f-827d-4a0a-941d-e327fad62be6 admin admin] PATH_INFO: `/v3/projects/ddb7a79ba6b74f698b26fd50db4a5345` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:58.711754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43fe91f-827d-4a0a-941d-e327fad62be6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddb7a79ba6b74f698b26fd50db4a5345)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:58.714914 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43fe91f-827d-4a0a-941d-e327fad62be6 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:58.746615 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:58.802758 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c43fe91f-827d-4a0a-941d-e327fad62be6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddb7a79ba6b74f698b26fd50db4a5345', 'request_id': 'req-c43fe91f-827d-4a0a-941d-e327fad62be6'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:58.814744 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c43fe91f-827d-4a0a-941d-e327fad62be6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddb7a79ba6b74f698b26fd50db4a5345', 'request_id': 'req-c43fe91f-827d-4a0a-941d-e327fad62be6'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:58.823481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c43fe91f-827d-4a0a-941d-e327fad62be6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddb7a79ba6b74f698b26fd50db4a5345', 'request_id': 'req-c43fe91f-827d-4a0a-941d-e327fad62be6'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:58.840477 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c43fe91f-827d-4a0a-941d-e327fad62be6 admin admin] The token cache is being invalidate because project ddb7a79ba6b74f698b26fd50db4a5345 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:56:58.840831 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c43fe91f-827d-4a0a-941d-e327fad62be6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:58.841813 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c43fe91f-827d-4a0a-941d-e327fad62be6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:58.843427 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 968/1951] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:56:58 2022] DELETE /identity/v3/projects/ddb7a79ba6b74f698b26fd50db4a5345 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:58.847707 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9571b5-7dd9-47c2-9cab-8d99de66650b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:58.849153 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4e9571b5-7dd9-47c2-9cab-8d99de66650b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:58.869727 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9571b5-7dd9-47c2-9cab-8d99de66650b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:58.956094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4b6f32-4459-472f-8171-c8df5bec4347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:58.957423 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b6f32-4459-472f-8171-c8df5bec4347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:58.957708 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b6f32-4459-472f-8171-c8df5bec4347 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:58.958035 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b6f32-4459-472f-8171-c8df5bec4347 admin admin] PATH_INFO: `/v3/users/f46ecf12cefe4c168f2e660cd0068da5` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:58.959311 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4b6f32-4459-472f-8171-c8df5bec4347 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f46ecf12cefe4c168f2e660cd0068da5)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:58.961659 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4b6f32-4459-472f-8171-c8df5bec4347 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:59.042465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2d4b6f32-4459-472f-8171-c8df5bec4347 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f46ecf12cefe4c168f2e660cd0068da5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:59.059330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2d4b6f32-4459-472f-8171-c8df5bec4347 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f46ecf12cefe4c168f2e660cd0068da5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:59.069965 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2d4b6f32-4459-472f-8171-c8df5bec4347 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f46ecf12cefe4c168f2e660cd0068da5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:59.070694 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2d4b6f32-4459-472f-8171-c8df5bec4347 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f46ecf12cefe4c168f2e660cd0068da5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:59.071464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2d4b6f32-4459-472f-8171-c8df5bec4347 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f46ecf12cefe4c168f2e660cd0068da5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:59.085891 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:59.093441 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 984/1952] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:56:58 2022] DELETE /identity/v3/users/f46ecf12cefe4c168f2e660cd0068da5 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:56:59.776015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43fe91f-827d-4a0a-941d-e327fad62be6 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:59.778074 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-c43fe91f-827d-4a0a-941d-e327fad62be6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:56:59.800021 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43fe91f-827d-4a0a-941d-e327fad62be6 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:59.886911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1117f-a93c-44ec-ac59-90bf40b68acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:59.887798 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1117f-a93c-44ec-ac59-90bf40b68acc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:59.888081 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1117f-a93c-44ec-ac59-90bf40b68acc admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:59.888359 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1117f-a93c-44ec-ac59-90bf40b68acc admin admin] PATH_INFO: `/v3/projects/c325af4399c2479faf47b377b927a667` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:59.889742 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d1117f-a93c-44ec-ac59-90bf40b68acc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c325af4399c2479faf47b377b927a667)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:59.892032 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d1117f-a93c-44ec-ac59-90bf40b68acc admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:59.965117 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-60d1117f-a93c-44ec-ac59-90bf40b68acc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c325af4399c2479faf47b377b927a667', 'request_id': 'req-60d1117f-a93c-44ec-ac59-90bf40b68acc'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:59.972988 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-60d1117f-a93c-44ec-ac59-90bf40b68acc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c325af4399c2479faf47b377b927a667', 'request_id': 'req-60d1117f-a93c-44ec-ac59-90bf40b68acc'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:56:59.982937 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-60d1117f-a93c-44ec-ac59-90bf40b68acc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c325af4399c2479faf47b377b927a667', 'request_id': 'req-60d1117f-a93c-44ec-ac59-90bf40b68acc'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:00.001537 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-60d1117f-a93c-44ec-ac59-90bf40b68acc admin admin] The token cache is being invalidate because project c325af4399c2479faf47b377b927a667 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:57:00.001842 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-60d1117f-a93c-44ec-ac59-90bf40b68acc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:00.002568 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-60d1117f-a93c-44ec-ac59-90bf40b68acc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:00.004287 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 969/1953] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:56:59 2022] DELETE /identity/v3/projects/c325af4399c2479faf47b377b927a667 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:00.011032 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36154a25-e79c-4372-81c0-4ddfd6d0c372 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:00.011345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36154a25-e79c-4372-81c0-4ddfd6d0c372 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:00.012126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36154a25-e79c-4372-81c0-4ddfd6d0c372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:00.031974 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-36154a25-e79c-4372-81c0-4ddfd6d0c372 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:00.033478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-36154a25-e79c-4372-81c0-4ddfd6d0c372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:00.077501 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1117f-a93c-44ec-ac59-90bf40b68acc admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:00.079710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-60d1117f-a93c-44ec-ac59-90bf40b68acc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:00.114402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 985/1954] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:57:00.120096 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871d7d1-184a-47f2-9b10-b374e5a375e3 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:00.120948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871d7d1-184a-47f2-9b10-b374e5a375e3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:00.121286 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871d7d1-184a-47f2-9b10-b374e5a375e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:00.143835 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-1871d7d1-184a-47f2-9b10-b374e5a375e3 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:00.145245 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1871d7d1-184a-47f2-9b10-b374e5a375e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:00.153940 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 986/1955] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:57:00.173584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871d7d1-184a-47f2-9b10-b374e5a375e3 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:00.185166 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:00.189392 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871d7d1-184a-47f2-9b10-b374e5a375e3 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:00.197220 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526fec6-fb76-4834-8dcf-95aa89840680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:00.199085 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526fec6-fb76-4834-8dcf-95aa89840680 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:00.199543 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526fec6-fb76-4834-8dcf-95aa89840680 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:00.200409 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526fec6-fb76-4834-8dcf-95aa89840680 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:00.201618 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526fec6-fb76-4834-8dcf-95aa89840680 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:57:00.202141 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526fec6-fb76-4834-8dcf-95aa89840680 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:00.205428 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526fec6-fb76-4834-8dcf-95aa89840680 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:00.219430 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1117f-a93c-44ec-ac59-90bf40b68acc admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:00.226396 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b27ff8-4007-4920-931a-522c5a0f699f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:00.227689 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b27ff8-4007-4920-931a-522c5a0f699f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:00.228034 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b27ff8-4007-4920-931a-522c5a0f699f admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:00.228373 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b27ff8-4007-4920-931a-522c5a0f699f admin admin] PATH_INFO: `/v3/users/e046ac3cf4da44cdaead991b6060f1f4` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:00.230004 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b27ff8-4007-4920-931a-522c5a0f699f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e046ac3cf4da44cdaead991b6060f1f4)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:00.232555 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b27ff8-4007-4920-931a-522c5a0f699f admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:00.243560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 987/1956] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:57:00.249124 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526fec6-fb76-4834-8dcf-95aa89840680 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:00.264592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526fec6-fb76-4834-8dcf-95aa89840680 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:00.270656 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a1d03-2c4a-41a5-b49a-32b9a2c197ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:00.271607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a1d03-2c4a-41a5-b49a-32b9a2c197ca admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:00.272137 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a1d03-2c4a-41a5-b49a-32b9a2c197ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:00.272570 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a1d03-2c4a-41a5-b49a-32b9a2c197ca admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:00.273432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a1d03-2c4a-41a5-b49a-32b9a2c197ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:00.275992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a1d03-2c4a-41a5-b49a-32b9a2c197ca admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:00.324567 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 988/1957] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:57:00 2022] POST /identity/v3/projects => generated 391 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:57:00.328234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a1d03-2c4a-41a5-b49a-32b9a2c197ca admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:00.350514 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-e6b27ff8-4007-4920-931a-522c5a0f699f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e046ac3cf4da44cdaead991b6060f1f4'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:00.385334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-e6b27ff8-4007-4920-931a-522c5a0f699f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e046ac3cf4da44cdaead991b6060f1f4'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:00.405341 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-e6b27ff8-4007-4920-931a-522c5a0f699f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e046ac3cf4da44cdaead991b6060f1f4'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:00.406149 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-e6b27ff8-4007-4920-931a-522c5a0f699f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e046ac3cf4da44cdaead991b6060f1f4'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:00.407050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-e6b27ff8-4007-4920-931a-522c5a0f699f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e046ac3cf4da44cdaead991b6060f1f4'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:00.431022 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 970/1958] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:57:00 2022] DELETE /identity/v3/users/e046ac3cf4da44cdaead991b6060f1f4 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:00.433616 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02713a4f-4a44-451c-ab9d-62991e5adf74 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:00.433995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02713a4f-4a44-451c-ab9d-62991e5adf74 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:00.435394 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02713a4f-4a44-451c-ab9d-62991e5adf74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:00.462220 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-02713a4f-4a44-451c-ab9d-62991e5adf74 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:00.463507 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-02713a4f-4a44-451c-ab9d-62991e5adf74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:00.480503 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a1d03-2c4a-41a5-b49a-32b9a2c197ca admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:00.552003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 971/1959] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:57:00.565201 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02713a4f-4a44-451c-ab9d-62991e5adf74 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:00.570456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-02713a4f-4a44-451c-ab9d-62991e5adf74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:00.585409 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfbd52-e762-437d-8580-5fc9ba5ff8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:00.587532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfbd52-e762-437d-8580-5fc9ba5ff8a2 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:00.588235 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfbd52-e762-437d-8580-5fc9ba5ff8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:00.589609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfbd52-e762-437d-8580-5fc9ba5ff8a2 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:00.591194 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfbd52-e762-437d-8580-5fc9ba5ff8a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:00.595983 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfbd52-e762-437d-8580-5fc9ba5ff8a2 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:00.614401 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:00.636310 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 989/1960] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:57:00 2022] POST /identity/v3/users => generated 296 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:57:00.640801 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfbd52-e762-437d-8580-5fc9ba5ff8a2 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:00.643178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-80dfbd52-e762-437d-8580-5fc9ba5ff8a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:00.662150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfbd52-e762-437d-8580-5fc9ba5ff8a2 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:00.667501 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca484eb1-cc59-440c-a51d-7049d9ecffa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:00.668789 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca484eb1-cc59-440c-a51d-7049d9ecffa5 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:00.668789 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca484eb1-cc59-440c-a51d-7049d9ecffa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:00.668986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca484eb1-cc59-440c-a51d-7049d9ecffa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:00.669797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca484eb1-cc59-440c-a51d-7049d9ecffa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:00.671657 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca484eb1-cc59-440c-a51d-7049d9ecffa5 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:00.684920 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 990/1961] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:57:00.690623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02713a4f-4a44-451c-ab9d-62991e5adf74 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:00.691116 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca484eb1-cc59-440c-a51d-7049d9ecffa5 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:00.696162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0ae00-ed0e-4dd6-9091-f1c1ec73fd9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:00.696901 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0ae00-ed0e-4dd6-9091-f1c1ec73fd9a service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:00.697322 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0ae00-ed0e-4dd6-9091-f1c1ec73fd9a service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:00.697700 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0ae00-ed0e-4dd6-9091-f1c1ec73fd9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:00.708944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0ae00-ed0e-4dd6-9091-f1c1ec73fd9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:00.710894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0ae00-ed0e-4dd6-9091-f1c1ec73fd9a service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:00.713605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca484eb1-cc59-440c-a51d-7049d9ecffa5 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:00.728270 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:00.732046 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d50f76-1bb8-494f-bb0f-50d51533443b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:00.732865 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d50f76-1bb8-494f-bb0f-50d51533443b admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:00.733154 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d50f76-1bb8-494f-bb0f-50d51533443b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:00.733495 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d50f76-1bb8-494f-bb0f-50d51533443b admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:00.734297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d50f76-1bb8-494f-bb0f-50d51533443b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:00.736046 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d50f76-1bb8-494f-bb0f-50d51533443b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:00.738754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 972/1962] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:57:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:57:00.758619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 991/1963] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:57:00.759795 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0ae00-ed0e-4dd6-9091-f1c1ec73fd9a service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:00.774261 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0ae00-ed0e-4dd6-9091-f1c1ec73fd9a service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:00.782872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bb83e-d54a-4ba4-b182-d999babcb966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:00.782872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bb83e-d54a-4ba4-b182-d999babcb966 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:00.783934 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bb83e-d54a-4ba4-b182-d999babcb966 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:00.783934 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bb83e-d54a-4ba4-b182-d999babcb966 admin admin] PATH_INFO: `/v3/projects/841fc05dee144faf98f863bcc2571d65/users/cdf7c8c021934d23bc480f80f32e8ccc/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:00.807003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80bb83e-d54a-4ba4-b182-d999babcb966 admin admin] RBAC: Authorizing `identity:create_grant(project_id=841fc05dee144faf98f863bcc2571d65, user_id=cdf7c8c021934d23bc480f80f32e8ccc, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:00.813415 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80bb83e-d54a-4ba4-b182-d999babcb966 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:00.855387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 973/1964] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:57:00 2022] PUT /identity/v3/projects/841fc05dee144faf98f863bcc2571d65/users/cdf7c8c021934d23bc480f80f32e8ccc/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:00.866925 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40376b4a-1e32-4cdd-873d-4b47247b3b86 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:00.867222 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40376b4a-1e32-4cdd-873d-4b47247b3b86 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:00.867509 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40376b4a-1e32-4cdd-873d-4b47247b3b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:00.919093 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-40376b4a-1e32-4cdd-873d-4b47247b3b86 None None] MFA Rules not processed for user `cdf7c8c021934d23bc480f80f32e8ccc`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:00.920640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-40376b4a-1e32-4cdd-873d-4b47247b3b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:01.037661 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bb83e-d54a-4ba4-b182-d999babcb966 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:01.038391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 992/1965] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:01.216153 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bb83e-d54a-4ba4-b182-d999babcb966 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:01.232078 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d92676-1fa8-4b06-b24c-1606ac429430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:01.233196 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d92676-1fa8-4b06-b24c-1606ac429430 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:01.233708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d92676-1fa8-4b06-b24c-1606ac429430 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:01.234249 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d92676-1fa8-4b06-b24c-1606ac429430 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:01.235881 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d92676-1fa8-4b06-b24c-1606ac429430 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:01.239494 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d92676-1fa8-4b06-b24c-1606ac429430 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:01.301631 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 974/1966] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:57:01 2022] POST /identity/v3/projects => generated 391 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:57:01.305881 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40376b4a-1e32-4cdd-873d-4b47247b3b86 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:01.318270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40376b4a-1e32-4cdd-873d-4b47247b3b86 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:01.408411 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48e0d5-ee70-44ef-8a73-e1cb975f28bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:01.409650 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48e0d5-ee70-44ef-8a73-e1cb975f28bb admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:01.410192 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48e0d5-ee70-44ef-8a73-e1cb975f28bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:01.410697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48e0d5-ee70-44ef-8a73-e1cb975f28bb admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:01.412067 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48e0d5-ee70-44ef-8a73-e1cb975f28bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:01.417031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48e0d5-ee70-44ef-8a73-e1cb975f28bb admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:01.463047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 993/1967] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:57:01 2022] POST /identity/v3/users => generated 302 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:57:01.465473 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d92676-1fa8-4b06-b24c-1606ac429430 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:01.493481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d92676-1fa8-4b06-b24c-1606ac429430 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:01.496159 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48e0d5-ee70-44ef-8a73-e1cb975f28bb admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:01.501549 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2e48e0d5-ee70-44ef-8a73-e1cb975f28bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:01.507968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c54a09-1bfa-4a3d-8244-d09cfecd7083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:01.509333 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c54a09-1bfa-4a3d-8244-d09cfecd7083 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:01.509333 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c54a09-1bfa-4a3d-8244-d09cfecd7083 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:01.509333 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c54a09-1bfa-4a3d-8244-d09cfecd7083 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:01.510161 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c54a09-1bfa-4a3d-8244-d09cfecd7083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:01.522510 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c54a09-1bfa-4a3d-8244-d09cfecd7083 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:01.541365 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 975/1968] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:57:01.553148 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48e0d5-ee70-44ef-8a73-e1cb975f28bb admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:01.554596 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c54a09-1bfa-4a3d-8244-d09cfecd7083 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:01.559865 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb4a2d-0a5a-4359-b8ee-815f44aab304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:01.561094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb4a2d-0a5a-4359-b8ee-815f44aab304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:01.561602 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb4a2d-0a5a-4359-b8ee-815f44aab304 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:01.562131 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb4a2d-0a5a-4359-b8ee-815f44aab304 admin admin] PATH_INFO: `/v3/projects/0259cabe6366480b874562418bd7ee32` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:01.567526 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb4a2d-0a5a-4359-b8ee-815f44aab304 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0259cabe6366480b874562418bd7ee32)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:01.575437 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c54a09-1bfa-4a3d-8244-d09cfecd7083 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:01.581076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb4a2d-0a5a-4359-b8ee-815f44aab304 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:01.581733 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad37fd7-d785-491e-b52d-fc415d9a29c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:01.583855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad37fd7-d785-491e-b52d-fc415d9a29c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:01.584123 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad37fd7-d785-491e-b52d-fc415d9a29c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:01.584407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad37fd7-d785-491e-b52d-fc415d9a29c5 admin admin] PATH_INFO: `/v3/projects/2602fa3d2ebf425ab6e0c34cb9dd6298/users/dae5b67c5951484180c8b5b25429bb86/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:01.607793 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad37fd7-d785-491e-b52d-fc415d9a29c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2602fa3d2ebf425ab6e0c34cb9dd6298, user_id=dae5b67c5951484180c8b5b25429bb86, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:01.611138 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad37fd7-d785-491e-b52d-fc415d9a29c5 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:01.630810 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 976/1969] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:57:01 2022] PUT /identity/v3/projects/2602fa3d2ebf425ab6e0c34cb9dd6298/users/dae5b67c5951484180c8b5b25429bb86/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:01.636848 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad37fd7-d785-491e-b52d-fc415d9a29c5 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:01.670632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:INSERT=4,keystone:SELECT=93,keystone:DELETE=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:01.674915 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-20eb4a2d-0a5a-4359-b8ee-815f44aab304 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0259cabe6366480b874562418bd7ee32', 'request_id': 'req-20eb4a2d-0a5a-4359-b8ee-815f44aab304'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:01.694869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-20eb4a2d-0a5a-4359-b8ee-815f44aab304 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0259cabe6366480b874562418bd7ee32', 'request_id': 'req-20eb4a2d-0a5a-4359-b8ee-815f44aab304'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:01.704572 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-20eb4a2d-0a5a-4359-b8ee-815f44aab304 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0259cabe6366480b874562418bd7ee32', 'request_id': 'req-20eb4a2d-0a5a-4359-b8ee-815f44aab304'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:01.722721 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-20eb4a2d-0a5a-4359-b8ee-815f44aab304 admin admin] The token cache is being invalidate because project 0259cabe6366480b874562418bd7ee32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:57:01.723112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-20eb4a2d-0a5a-4359-b8ee-815f44aab304 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:01.724023 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-20eb4a2d-0a5a-4359-b8ee-815f44aab304 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:01.725719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 994/1970] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:57:01 2022] DELETE /identity/v3/projects/0259cabe6366480b874562418bd7ee32 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:01.730678 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb4a2d-0a5a-4359-b8ee-815f44aab304 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:01.733001 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-20eb4a2d-0a5a-4359-b8ee-815f44aab304 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:01.742935 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad37fd7-d785-491e-b52d-fc415d9a29c5 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:01.786377 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:01.827112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bdce1-f558-43be-bba9-b527268fcce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:01.828096 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bdce1-f558-43be-bba9-b527268fcce4 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:01.828543 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bdce1-f558-43be-bba9-b527268fcce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:01.829004 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bdce1-f558-43be-bba9-b527268fcce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:01.829758 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818bdce1-f558-43be-bba9-b527268fcce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:01.831088 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb4a2d-0a5a-4359-b8ee-815f44aab304 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:01.831451 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818bdce1-f558-43be-bba9-b527268fcce4 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:01.836446 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349dd85-87c7-42a8-9f73-dfef952688cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:01.837559 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349dd85-87c7-42a8-9f73-dfef952688cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:01.837846 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349dd85-87c7-42a8-9f73-dfef952688cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:01.838143 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349dd85-87c7-42a8-9f73-dfef952688cf admin admin] PATH_INFO: `/v3/users/2ee3e13df6d443ce97139c5753885877` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:01.841225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349dd85-87c7-42a8-9f73-dfef952688cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ee3e13df6d443ce97139c5753885877)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:01.849758 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 977/1971] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:57:01.849973 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349dd85-87c7-42a8-9f73-dfef952688cf admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:01.852199 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bdce1-f558-43be-bba9-b527268fcce4 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:01.854206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-818bdce1-f558-43be-bba9-b527268fcce4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:01.879751 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bdce1-f558-43be-bba9-b527268fcce4 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:01.892166 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6ecbb-3ec8-46bd-9e8a-9e59fa357c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:01.893672 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6ecbb-3ec8-46bd-9e8a-9e59fa357c4b admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:01.894965 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6ecbb-3ec8-46bd-9e8a-9e59fa357c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:01.896969 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6ecbb-3ec8-46bd-9e8a-9e59fa357c4b admin admin] PATH_INFO: `/v3/projects/2602fa3d2ebf425ab6e0c34cb9dd6298/users/dae5b67c5951484180c8b5b25429bb86/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:01.906157 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d6ecbb-3ec8-46bd-9e8a-9e59fa357c4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2602fa3d2ebf425ab6e0c34cb9dd6298, user_id=dae5b67c5951484180c8b5b25429bb86, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:01.906157 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d6ecbb-3ec8-46bd-9e8a-9e59fa357c4b admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:01.937192 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 978/1972] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:57:01 2022] PUT /identity/v3/projects/2602fa3d2ebf425ab6e0c34cb9dd6298/users/dae5b67c5951484180c8b5b25429bb86/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:01.954233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c3df4-ddf0-496f-ad6e-88e604bc0634 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:01.954638 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c3df4-ddf0-496f-ad6e-88e604bc0634 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:01.955161 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c3df4-ddf0-496f-ad6e-88e604bc0634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:01.973310 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e349dd85-87c7-42a8-9f73-dfef952688cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ee3e13df6d443ce97139c5753885877'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:01.994947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e349dd85-87c7-42a8-9f73-dfef952688cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ee3e13df6d443ce97139c5753885877'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:02.000247 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-551c3df4-ddf0-496f-ad6e-88e604bc0634 None None] MFA Rules not processed for user `dae5b67c5951484180c8b5b25429bb86`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:02.001487 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-551c3df4-ddf0-496f-ad6e-88e604bc0634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:02.007505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e349dd85-87c7-42a8-9f73-dfef952688cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ee3e13df6d443ce97139c5753885877'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:02.008006 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e349dd85-87c7-42a8-9f73-dfef952688cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ee3e13df6d443ce97139c5753885877'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:02.008774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e349dd85-87c7-42a8-9f73-dfef952688cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ee3e13df6d443ce97139c5753885877'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:02.034945 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 995/1973] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:57:01 2022] DELETE /identity/v3/users/2ee3e13df6d443ce97139c5753885877 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:57:02.107342 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 979/1974] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:01 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:02.167978 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349dd85-87c7-42a8-9f73-dfef952688cf admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:02.171824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e349dd85-87c7-42a8-9f73-dfef952688cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:02.300905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349dd85-87c7-42a8-9f73-dfef952688cf admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:02.306863 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae522f1-560e-4238-b83f-075cbe140272 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:02.308530 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae522f1-560e-4238-b83f-075cbe140272 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:02.309111 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae522f1-560e-4238-b83f-075cbe140272 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:02.309740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae522f1-560e-4238-b83f-075cbe140272 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:02.318379 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:02.421353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c3df4-ddf0-496f-ad6e-88e604bc0634 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:02.423709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-551c3df4-ddf0-496f-ad6e-88e604bc0634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:02.457253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae522f1-560e-4238-b83f-075cbe140272 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:02.462063 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae522f1-560e-4238-b83f-075cbe140272 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:02.474338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 996/1975] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:57:02 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:57:02.598522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c3df4-ddf0-496f-ad6e-88e604bc0634 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:02.602808 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae522f1-560e-4238-b83f-075cbe140272 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:02.612525 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae522f1-560e-4238-b83f-075cbe140272 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:02.617109 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44169ad2-618a-447e-aec1-a6089aa8e1bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:02.617885 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44169ad2-618a-447e-aec1-a6089aa8e1bd service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:02.618511 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44169ad2-618a-447e-aec1-a6089aa8e1bd service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:02.618830 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44169ad2-618a-447e-aec1-a6089aa8e1bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:02.621972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-44169ad2-618a-447e-aec1-a6089aa8e1bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:02.675148 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:02.693484 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d22c19-4049-483f-aa60-29a189021972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:02.695523 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d22c19-4049-483f-aa60-29a189021972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:02.695523 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d22c19-4049-483f-aa60-29a189021972 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:02.695523 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d22c19-4049-483f-aa60-29a189021972 admin admin] PATH_INFO: `/v3/projects/6b6dc6bd6ec04d30875c69c354ccc219` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:02.697723 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d22c19-4049-483f-aa60-29a189021972 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b6dc6bd6ec04d30875c69c354ccc219)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:02.700603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d22c19-4049-483f-aa60-29a189021972 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:02.712794 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44169ad2-618a-447e-aec1-a6089aa8e1bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:02.714861 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44169ad2-618a-447e-aec1-a6089aa8e1bd service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:02.730697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 997/1976] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:57:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:57:02.786252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d9d22c19-4049-483f-aa60-29a189021972 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b6dc6bd6ec04d30875c69c354ccc219', 'request_id': 'req-d9d22c19-4049-483f-aa60-29a189021972'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:02.797442 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d9d22c19-4049-483f-aa60-29a189021972 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b6dc6bd6ec04d30875c69c354ccc219', 'request_id': 'req-d9d22c19-4049-483f-aa60-29a189021972'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:02.806959 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d9d22c19-4049-483f-aa60-29a189021972 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b6dc6bd6ec04d30875c69c354ccc219', 'request_id': 'req-d9d22c19-4049-483f-aa60-29a189021972'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:02.830509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d9d22c19-4049-483f-aa60-29a189021972 admin admin] The token cache is being invalidate because project 6b6dc6bd6ec04d30875c69c354ccc219 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:57:02.830834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d9d22c19-4049-483f-aa60-29a189021972 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:02.832036 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d9d22c19-4049-483f-aa60-29a189021972 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:02.833725 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 980/1977] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:57:02 2022] DELETE /identity/v3/projects/6b6dc6bd6ec04d30875c69c354ccc219 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:57:02.838784 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9d1ae-7034-40c3-978d-7cfc9ee136bf None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:02.839083 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9d1ae-7034-40c3-978d-7cfc9ee136bf None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:02.839367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9d1ae-7034-40c3-978d-7cfc9ee136bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:02.860537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-ccd9d1ae-7034-40c3-978d-7cfc9ee136bf None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:02.861778 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ccd9d1ae-7034-40c3-978d-7cfc9ee136bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:02.945987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 998/1978] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:02.953993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de534524-801e-48b4-83fc-749d2a392ee0 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:02.954314 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de534524-801e-48b4-83fc-749d2a392ee0 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:02.954597 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de534524-801e-48b4-83fc-749d2a392ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:02.991037 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-de534524-801e-48b4-83fc-749d2a392ee0 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:02.992703 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-de534524-801e-48b4-83fc-749d2a392ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:03.006051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 981/1979] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:03.009294 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9d1ae-7034-40c3-978d-7cfc9ee136bf None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:03.020425 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9d1ae-7034-40c3-978d-7cfc9ee136bf None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:03.025785 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3c8d7-0aa3-4e15-a4f3-b8e0a58af4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:03.026667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3c8d7-0aa3-4e15-a4f3-b8e0a58af4f9 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:03.027014 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3c8d7-0aa3-4e15-a4f3-b8e0a58af4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:03.027241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3c8d7-0aa3-4e15-a4f3-b8e0a58af4f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:03.028417 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3c8d7-0aa3-4e15-a4f3-b8e0a58af4f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:57:03.028417 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3c8d7-0aa3-4e15-a4f3-b8e0a58af4f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:03.030400 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3c8d7-0aa3-4e15-a4f3-b8e0a58af4f9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:03.054865 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 999/1980] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:57:03.055712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de534524-801e-48b4-83fc-749d2a392ee0 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:03.069750 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de534524-801e-48b4-83fc-749d2a392ee0 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:03.074882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d7b30-0b87-4fc1-8aa3-f02f93dfe1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:03.075617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d7b30-0b87-4fc1-8aa3-f02f93dfe1f0 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:03.075970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d7b30-0b87-4fc1-8aa3-f02f93dfe1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:03.076314 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d7b30-0b87-4fc1-8aa3-f02f93dfe1f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:03.077151 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d7b30-0b87-4fc1-8aa3-f02f93dfe1f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:03.088810 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d7b30-0b87-4fc1-8aa3-f02f93dfe1f0 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:03.136040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 982/1981] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:57:03 2022] POST /identity/v3/projects => generated 407 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:57:03.138503 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3c8d7-0aa3-4e15-a4f3-b8e0a58af4f9 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:03.156537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3c8d7-0aa3-4e15-a4f3-b8e0a58af4f9 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:03.199011 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:03.245587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30599daa-dcf5-489d-817c-726f1c976716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:03.246738 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30599daa-dcf5-489d-817c-726f1c976716 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:03.247183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30599daa-dcf5-489d-817c-726f1c976716 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:03.247602 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30599daa-dcf5-489d-817c-726f1c976716 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:03.249043 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30599daa-dcf5-489d-817c-726f1c976716 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:03.252007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30599daa-dcf5-489d-817c-726f1c976716 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:03.276837 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1000/1982] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:57:03 2022] POST /identity/v3/users => generated 304 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:57:03.278118 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d7b30-0b87-4fc1-8aa3-f02f93dfe1f0 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:03.290050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d7b30-0b87-4fc1-8aa3-f02f93dfe1f0 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:03.295801 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae034fc-22d1-49ff-add9-78400a2c07a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:03.296467 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae034fc-22d1-49ff-add9-78400a2c07a2 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:03.296751 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae034fc-22d1-49ff-add9-78400a2c07a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:03.297028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae034fc-22d1-49ff-add9-78400a2c07a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:03.297849 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae034fc-22d1-49ff-add9-78400a2c07a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:03.299419 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae034fc-22d1-49ff-add9-78400a2c07a2 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:03.310614 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 983/1983] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:57:03.315083 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30599daa-dcf5-489d-817c-726f1c976716 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:03.328130 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30599daa-dcf5-489d-817c-726f1c976716 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:03.336209 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55810d-6840-4fc9-bd67-d6cfe3210e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:03.336889 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55810d-6840-4fc9-bd67-d6cfe3210e5c admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:03.337175 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55810d-6840-4fc9-bd67-d6cfe3210e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:03.337453 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55810d-6840-4fc9-bd67-d6cfe3210e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:03.338239 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f55810d-6840-4fc9-bd67-d6cfe3210e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:03.340418 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f55810d-6840-4fc9-bd67-d6cfe3210e5c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:03.351925 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1001/1984] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:57:03.357331 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae034fc-22d1-49ff-add9-78400a2c07a2 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:03.368948 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae034fc-22d1-49ff-add9-78400a2c07a2 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:03.373688 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b685128-2d60-45e1-b118-8c9a2ca44a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:03.374613 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b685128-2d60-45e1-b118-8c9a2ca44a7b admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:03.374895 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b685128-2d60-45e1-b118-8c9a2ca44a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:03.375165 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b685128-2d60-45e1-b118-8c9a2ca44a7b admin admin] PATH_INFO: `/v3/projects/2b000017a4144a188bbf1649b73df0a2/users/4e29516f01f849bd8632af7bcff7ee28/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:03.390332 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b685128-2d60-45e1-b118-8c9a2ca44a7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b000017a4144a188bbf1649b73df0a2, user_id=4e29516f01f849bd8632af7bcff7ee28, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:03.393042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b685128-2d60-45e1-b118-8c9a2ca44a7b admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:03.407096 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 984/1985] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:57:03 2022] PUT /identity/v3/projects/2b000017a4144a188bbf1649b73df0a2/users/4e29516f01f849bd8632af7bcff7ee28/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:03.417655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57328d3-8735-43d6-be5a-6f3fdcbc0d62 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:03.417950 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57328d3-8735-43d6-be5a-6f3fdcbc0d62 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:03.418255 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57328d3-8735-43d6-be5a-6f3fdcbc0d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:03.445058 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-f57328d3-8735-43d6-be5a-6f3fdcbc0d62 None None] MFA Rules not processed for user `4e29516f01f849bd8632af7bcff7ee28`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:03.446289 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f57328d3-8735-43d6-be5a-6f3fdcbc0d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:03.511704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1002/1986] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:03.516135 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b685128-2d60-45e1-b118-8c9a2ca44a7b admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:03.577646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b685128-2d60-45e1-b118-8c9a2ca44a7b admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:03.582203 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59ea98-0ad8-48b6-b2f1-7e8cfea5fd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:03.582849 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59ea98-0ad8-48b6-b2f1-7e8cfea5fd46 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:03.583126 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59ea98-0ad8-48b6-b2f1-7e8cfea5fd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:03.583396 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59ea98-0ad8-48b6-b2f1-7e8cfea5fd46 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:03.584113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59ea98-0ad8-48b6-b2f1-7e8cfea5fd46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:03.586181 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59ea98-0ad8-48b6-b2f1-7e8cfea5fd46 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:03.608335 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:03.616695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 985/1987] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:57:03 2022] POST /identity/v3/projects => generated 409 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:57:03.620456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57328d3-8735-43d6-be5a-6f3fdcbc0d62 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:03.632224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57328d3-8735-43d6-be5a-6f3fdcbc0d62 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:03.701946 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eead745-534f-439d-81fa-b3364bb0ec07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:03.703018 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eead745-534f-439d-81fa-b3364bb0ec07 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:03.703018 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eead745-534f-439d-81fa-b3364bb0ec07 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:03.703248 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eead745-534f-439d-81fa-b3364bb0ec07 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:03.703878 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eead745-534f-439d-81fa-b3364bb0ec07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:03.706658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eead745-534f-439d-81fa-b3364bb0ec07 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:03.737284 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1003/1988] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:57:03 2022] POST /identity/v3/users => generated 311 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:57:03.741807 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59ea98-0ad8-48b6-b2f1-7e8cfea5fd46 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:03.753421 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59ea98-0ad8-48b6-b2f1-7e8cfea5fd46 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:03.758083 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891607c-d046-4103-b1d2-e642b7f2f5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:03.758747 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891607c-d046-4103-b1d2-e642b7f2f5b8 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:03.759020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891607c-d046-4103-b1d2-e642b7f2f5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:03.759285 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891607c-d046-4103-b1d2-e642b7f2f5b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:03.760033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5891607c-d046-4103-b1d2-e642b7f2f5b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:03.762186 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5891607c-d046-4103-b1d2-e642b7f2f5b8 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:03.773314 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 986/1989] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:57:03.778724 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eead745-534f-439d-81fa-b3364bb0ec07 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:03.789584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eead745-534f-439d-81fa-b3364bb0ec07 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:03.795105 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9acb82-2efb-4b02-8132-625b785a3f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:03.795995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9acb82-2efb-4b02-8132-625b785a3f0b admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:03.796278 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9acb82-2efb-4b02-8132-625b785a3f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:03.796556 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9acb82-2efb-4b02-8132-625b785a3f0b admin admin] PATH_INFO: `/v3/projects/901b7bea54a54497bc3cf53e7430d4ea/users/9f5601f539fc429984047e6b7268ae01/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:03.812468 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9acb82-2efb-4b02-8132-625b785a3f0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=901b7bea54a54497bc3cf53e7430d4ea, user_id=9f5601f539fc429984047e6b7268ae01, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:03.815384 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9acb82-2efb-4b02-8132-625b785a3f0b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:03.830541 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1004/1990] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:57:03 2022] PUT /identity/v3/projects/901b7bea54a54497bc3cf53e7430d4ea/users/9f5601f539fc429984047e6b7268ae01/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:03.834990 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891607c-d046-4103-b1d2-e642b7f2f5b8 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:03.844645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891607c-d046-4103-b1d2-e642b7f2f5b8 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:03.912300 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f916f-e4c2-484f-bc88-a079d7df076d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:03.913021 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f916f-e4c2-484f-bc88-a079d7df076d admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:03.913254 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f916f-e4c2-484f-bc88-a079d7df076d admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:03.913527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f916f-e4c2-484f-bc88-a079d7df076d admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:03.914624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f916f-e4c2-484f-bc88-a079d7df076d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:03.916150 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f916f-e4c2-484f-bc88-a079d7df076d admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:03.930083 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 987/1991] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:57:03.931572 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9acb82-2efb-4b02-8132-625b785a3f0b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:03.937811 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:03.944794 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9acb82-2efb-4b02-8132-625b785a3f0b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:03.950109 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c7ece-b1e3-4d7c-8bbe-1ec38f988826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:03.951116 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c7ece-b1e3-4d7c-8bbe-1ec38f988826 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:03.951475 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c7ece-b1e3-4d7c-8bbe-1ec38f988826 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:03.951825 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c7ece-b1e3-4d7c-8bbe-1ec38f988826 admin admin] PATH_INFO: `/v3/projects/901b7bea54a54497bc3cf53e7430d4ea/users/9f5601f539fc429984047e6b7268ae01/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:03.953843 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c7ece-b1e3-4d7c-8bbe-1ec38f988826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=901b7bea54a54497bc3cf53e7430d4ea, user_id=9f5601f539fc429984047e6b7268ae01, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:03.956574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c7ece-b1e3-4d7c-8bbe-1ec38f988826 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:03.967313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1005/1992] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:57:03 2022] PUT /identity/v3/projects/901b7bea54a54497bc3cf53e7430d4ea/users/9f5601f539fc429984047e6b7268ae01/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:03.977519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b69986-bf47-4ce3-96fe-436f4daf93ab None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:03.977874 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b69986-bf47-4ce3-96fe-436f4daf93ab None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:03.978248 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b69986-bf47-4ce3-96fe-436f4daf93ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:04.006651 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-a1b69986-bf47-4ce3-96fe-436f4daf93ab None None] MFA Rules not processed for user `9f5601f539fc429984047e6b7268ae01`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:04.007868 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-a1b69986-bf47-4ce3-96fe-436f4daf93ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:04.083571 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 988/1993] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:04.094633 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c7ece-b1e3-4d7c-8bbe-1ec38f988826 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:04.096408 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-dd5c7ece-b1e3-4d7c-8bbe-1ec38f988826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:04.159652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c7ece-b1e3-4d7c-8bbe-1ec38f988826 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:04.164274 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc5f6a-83b0-49f4-8922-3e00f87820ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:04.165019 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc5f6a-83b0-49f4-8922-3e00f87820ba service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:04.165369 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc5f6a-83b0-49f4-8922-3e00f87820ba service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:04.165711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc5f6a-83b0-49f4-8922-3e00f87820ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:04.177876 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc5f6a-83b0-49f4-8922-3e00f87820ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:04.179575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc5f6a-83b0-49f4-8922-3e00f87820ba service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:04.191461 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1006/1994] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:57:04 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:57:04.346795 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b69986-bf47-4ce3-96fe-436f4daf93ab None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:04.361280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b69986-bf47-4ce3-96fe-436f4daf93ab None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:04.365355 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c691e-429d-4278-aa22-2bb4c0c29a0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:04.366120 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c691e-429d-4278-aa22-2bb4c0c29a0e service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:04.366400 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c691e-429d-4278-aa22-2bb4c0c29a0e service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:04.366678 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c691e-429d-4278-aa22-2bb4c0c29a0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:04.425042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09c691e-429d-4278-aa22-2bb4c0c29a0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:04.426504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09c691e-429d-4278-aa22-2bb4c0c29a0e service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:04.435971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 989/1995] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:57:04 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:57:14.185425 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:14.431470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=96 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:25.214329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc5f6a-83b0-49f4-8922-3e00f87820ba service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:25.216515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c6cc5f6a-83b0-49f4-8922-3e00f87820ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:25.359137 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc5f6a-83b0-49f4-8922-3e00f87820ba service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:25.365566 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f124865-58fb-488e-8033-becea1268546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:25.366740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f124865-58fb-488e-8033-becea1268546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:25.367166 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f124865-58fb-488e-8033-becea1268546 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:25.367526 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f124865-58fb-488e-8033-becea1268546 admin admin] PATH_INFO: `/v3/users/cdf7c8c021934d23bc480f80f32e8ccc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:25.368856 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f124865-58fb-488e-8033-becea1268546 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdf7c8c021934d23bc480f80f32e8ccc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:25.370929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f124865-58fb-488e-8033-becea1268546 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:25.418078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5f124865-58fb-488e-8033-becea1268546 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdf7c8c021934d23bc480f80f32e8ccc'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:25.430785 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5f124865-58fb-488e-8033-becea1268546 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdf7c8c021934d23bc480f80f32e8ccc'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:25.438285 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5f124865-58fb-488e-8033-becea1268546 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdf7c8c021934d23bc480f80f32e8ccc'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:25.438921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5f124865-58fb-488e-8033-becea1268546 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdf7c8c021934d23bc480f80f32e8ccc'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:25.439559 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5f124865-58fb-488e-8033-becea1268546 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdf7c8c021934d23bc480f80f32e8ccc'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:25.454715 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1007/1996] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:57:25 2022] DELETE /identity/v3/users/cdf7c8c021934d23bc480f80f32e8ccc => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:25.457462 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a265ed-9816-47cf-ac6e-66ad62333881 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:25.458158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a265ed-9816-47cf-ac6e-66ad62333881 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:25.458158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a265ed-9816-47cf-ac6e-66ad62333881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:25.490308 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-82a265ed-9816-47cf-ac6e-66ad62333881 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:25.491630 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-82a265ed-9816-47cf-ac6e-66ad62333881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:25.556487 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 990/1997] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:25.566298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f124865-58fb-488e-8033-becea1268546 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:25.567774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5f124865-58fb-488e-8033-becea1268546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:25.592840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:25.670225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f124865-58fb-488e-8033-becea1268546 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:25.673884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ef9a6-0cb3-46ee-93cf-6198c5242e90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:25.674614 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ef9a6-0cb3-46ee-93cf-6198c5242e90 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:25.674937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ef9a6-0cb3-46ee-93cf-6198c5242e90 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:25.675218 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ef9a6-0cb3-46ee-93cf-6198c5242e90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:25.684523 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ef9a6-0cb3-46ee-93cf-6198c5242e90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:25.685955 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ef9a6-0cb3-46ee-93cf-6198c5242e90 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:25.695400 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1008/1998] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:57:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:57:26.018202 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a265ed-9816-47cf-ac6e-66ad62333881 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:26.019658 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-82a265ed-9816-47cf-ac6e-66ad62333881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:26.032175 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a265ed-9816-47cf-ac6e-66ad62333881 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:26.036983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda185e6-22c6-41f0-bdf6-a9fba01fa0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:26.037856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda185e6-22c6-41f0-bdf6-a9fba01fa0a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:26.038208 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda185e6-22c6-41f0-bdf6-a9fba01fa0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:26.038428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda185e6-22c6-41f0-bdf6-a9fba01fa0a3 admin admin] PATH_INFO: `/v3/projects/841fc05dee144faf98f863bcc2571d65` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:26.039651 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda185e6-22c6-41f0-bdf6-a9fba01fa0a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=841fc05dee144faf98f863bcc2571d65)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:26.041568 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda185e6-22c6-41f0-bdf6-a9fba01fa0a3 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:26.090872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-dda185e6-22c6-41f0-bdf6-a9fba01fa0a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '841fc05dee144faf98f863bcc2571d65', 'request_id': 'req-dda185e6-22c6-41f0-bdf6-a9fba01fa0a3'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:26.096597 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-dda185e6-22c6-41f0-bdf6-a9fba01fa0a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '841fc05dee144faf98f863bcc2571d65', 'request_id': 'req-dda185e6-22c6-41f0-bdf6-a9fba01fa0a3'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:26.103430 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-dda185e6-22c6-41f0-bdf6-a9fba01fa0a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '841fc05dee144faf98f863bcc2571d65', 'request_id': 'req-dda185e6-22c6-41f0-bdf6-a9fba01fa0a3'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:26.118211 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-dda185e6-22c6-41f0-bdf6-a9fba01fa0a3 admin admin] The token cache is being invalidate because project 841fc05dee144faf98f863bcc2571d65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:57:26.118505 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-dda185e6-22c6-41f0-bdf6-a9fba01fa0a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:26.119162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-dda185e6-22c6-41f0-bdf6-a9fba01fa0a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:26.121039 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 991/1999] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:57:26 2022] DELETE /identity/v3/projects/841fc05dee144faf98f863bcc2571d65 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:26.124773 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ef9a6-0cb3-46ee-93cf-6198c5242e90 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:26.138212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ef9a6-0cb3-46ee-93cf-6198c5242e90 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:26.222518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5271b0-c76a-4cfe-bbc5-34644e9d9e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:26.223547 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5271b0-c76a-4cfe-bbc5-34644e9d9e5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:26.223841 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5271b0-c76a-4cfe-bbc5-34644e9d9e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:26.224121 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5271b0-c76a-4cfe-bbc5-34644e9d9e5e admin admin] PATH_INFO: `/v3/users/dae5b67c5951484180c8b5b25429bb86` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:26.225342 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5271b0-c76a-4cfe-bbc5-34644e9d9e5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=dae5b67c5951484180c8b5b25429bb86)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:26.227340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5271b0-c76a-4cfe-bbc5-34644e9d9e5e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:26.290240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2c5271b0-c76a-4cfe-bbc5-34644e9d9e5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dae5b67c5951484180c8b5b25429bb86'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:26.306289 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2c5271b0-c76a-4cfe-bbc5-34644e9d9e5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dae5b67c5951484180c8b5b25429bb86'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:26.315561 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2c5271b0-c76a-4cfe-bbc5-34644e9d9e5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dae5b67c5951484180c8b5b25429bb86'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:26.316482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2c5271b0-c76a-4cfe-bbc5-34644e9d9e5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dae5b67c5951484180c8b5b25429bb86'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:26.317498 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2c5271b0-c76a-4cfe-bbc5-34644e9d9e5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dae5b67c5951484180c8b5b25429bb86'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:26.334875 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1009/2000] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:57:26 2022] DELETE /identity/v3/users/dae5b67c5951484180c8b5b25429bb86 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:26.409489 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda185e6-22c6-41f0-bdf6-a9fba01fa0a3 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:26.411515 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-dda185e6-22c6-41f0-bdf6-a9fba01fa0a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:26.425500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda185e6-22c6-41f0-bdf6-a9fba01fa0a3 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:26.520437 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09258790-8fc0-4b11-8e53-000155141e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:26.521515 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09258790-8fc0-4b11-8e53-000155141e9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:26.522299 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09258790-8fc0-4b11-8e53-000155141e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:26.522649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09258790-8fc0-4b11-8e53-000155141e9e admin admin] PATH_INFO: `/v3/users/4e29516f01f849bd8632af7bcff7ee28` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:26.524076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09258790-8fc0-4b11-8e53-000155141e9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e29516f01f849bd8632af7bcff7ee28)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:26.527467 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09258790-8fc0-4b11-8e53-000155141e9e admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:26.554660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:26.591969 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-09258790-8fc0-4b11-8e53-000155141e9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e29516f01f849bd8632af7bcff7ee28'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:26.605385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-09258790-8fc0-4b11-8e53-000155141e9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e29516f01f849bd8632af7bcff7ee28'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:26.614825 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-09258790-8fc0-4b11-8e53-000155141e9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e29516f01f849bd8632af7bcff7ee28'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:26.615465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-09258790-8fc0-4b11-8e53-000155141e9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e29516f01f849bd8632af7bcff7ee28'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:26.616073 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-09258790-8fc0-4b11-8e53-000155141e9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e29516f01f849bd8632af7bcff7ee28'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:26.629537 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 992/2001] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:57:26 2022] DELETE /identity/v3/users/4e29516f01f849bd8632af7bcff7ee28 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:26.634148 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016f370-d3ce-490a-917c-016e57f073d3 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:26.634445 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016f370-d3ce-490a-917c-016e57f073d3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:26.634730 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016f370-d3ce-490a-917c-016e57f073d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:26.641535 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:26.655404 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-9016f370-d3ce-490a-917c-016e57f073d3 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:26.657149 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9016f370-d3ce-490a-917c-016e57f073d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:26.689894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09258790-8fc0-4b11-8e53-000155141e9e admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:26.691550 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-09258790-8fc0-4b11-8e53-000155141e9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:26.746052 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1010/2002] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:26.755515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016f370-d3ce-490a-917c-016e57f073d3 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:26.757508 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9016f370-d3ce-490a-917c-016e57f073d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:26.773036 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09258790-8fc0-4b11-8e53-000155141e9e admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:26.778662 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96696fb-bc6c-4256-9c32-5a78cc26ca23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:26.779544 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96696fb-bc6c-4256-9c32-5a78cc26ca23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:26.779827 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96696fb-bc6c-4256-9c32-5a78cc26ca23 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:26.780101 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96696fb-bc6c-4256-9c32-5a78cc26ca23 admin admin] PATH_INFO: `/v3/projects/2602fa3d2ebf425ab6e0c34cb9dd6298` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:26.781320 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96696fb-bc6c-4256-9c32-5a78cc26ca23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2602fa3d2ebf425ab6e0c34cb9dd6298)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:26.783282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96696fb-bc6c-4256-9c32-5a78cc26ca23 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:26.816628 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016f370-d3ce-490a-917c-016e57f073d3 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:26.820259 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f93ba-2852-4cc4-b31f-74a321faf13a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:26.820909 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f93ba-2852-4cc4-b31f-74a321faf13a service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:26.821191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f93ba-2852-4cc4-b31f-74a321faf13a service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:26.821468 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f93ba-2852-4cc4-b31f-74a321faf13a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:26.831454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f93ba-2852-4cc4-b31f-74a321faf13a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:26.832755 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c96696fb-bc6c-4256-9c32-5a78cc26ca23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2602fa3d2ebf425ab6e0c34cb9dd6298', 'request_id': 'req-c96696fb-bc6c-4256-9c32-5a78cc26ca23'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:26.833143 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f93ba-2852-4cc4-b31f-74a321faf13a service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:26.840002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c96696fb-bc6c-4256-9c32-5a78cc26ca23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2602fa3d2ebf425ab6e0c34cb9dd6298', 'request_id': 'req-c96696fb-bc6c-4256-9c32-5a78cc26ca23'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:26.843817 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1011/2003] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:57:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:57:26.849300 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c96696fb-bc6c-4256-9c32-5a78cc26ca23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2602fa3d2ebf425ab6e0c34cb9dd6298', 'request_id': 'req-c96696fb-bc6c-4256-9c32-5a78cc26ca23'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:26.864935 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c96696fb-bc6c-4256-9c32-5a78cc26ca23 admin admin] The token cache is being invalidate because project 2602fa3d2ebf425ab6e0c34cb9dd6298 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:57:26.865304 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c96696fb-bc6c-4256-9c32-5a78cc26ca23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:26.866098 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c96696fb-bc6c-4256-9c32-5a78cc26ca23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:26.867748 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 993/2004] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:57:26 2022] DELETE /identity/v3/projects/2602fa3d2ebf425ab6e0c34cb9dd6298 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:57:26.872706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8784a2-2096-4ee8-9032-760ed20d5929 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:26.873009 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8784a2-2096-4ee8-9032-760ed20d5929 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:26.873292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8784a2-2096-4ee8-9032-760ed20d5929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:26.916390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-ca8784a2-2096-4ee8-9032-760ed20d5929 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:26.918205 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ca8784a2-2096-4ee8-9032-760ed20d5929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:26.982742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1012/2005] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:26.990113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c228a21-4ab3-48d4-bf46-a2d0767851ee None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:26.990408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c228a21-4ab3-48d4-bf46-a2d0767851ee None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:26.990686 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c228a21-4ab3-48d4-bf46-a2d0767851ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:27.009697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-7c228a21-4ab3-48d4-bf46-a2d0767851ee None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:27.011481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-7c228a21-4ab3-48d4-bf46-a2d0767851ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:27.020782 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 994/2006] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:57:27.024992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8784a2-2096-4ee8-9032-760ed20d5929 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:27.035437 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8784a2-2096-4ee8-9032-760ed20d5929 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:27.040392 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b9cd34-54f5-488d-8875-57d364948591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:27.041066 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9cd34-54f5-488d-8875-57d364948591 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:27.041350 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9cd34-54f5-488d-8875-57d364948591 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:27.041630 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9cd34-54f5-488d-8875-57d364948591 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:27.042938 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b9cd34-54f5-488d-8875-57d364948591 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:57:27.043250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b9cd34-54f5-488d-8875-57d364948591 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:27.044887 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b9cd34-54f5-488d-8875-57d364948591 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:27.060220 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1013/2007] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:57:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:57:27.061360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c228a21-4ab3-48d4-bf46-a2d0767851ee None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:27.080120 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c228a21-4ab3-48d4-bf46-a2d0767851ee None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:27.089081 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad883bb-4f72-4a15-818f-7ecf82b702b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:27.090195 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad883bb-4f72-4a15-818f-7ecf82b702b3 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:27.090360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad883bb-4f72-4a15-818f-7ecf82b702b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:27.090741 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad883bb-4f72-4a15-818f-7ecf82b702b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:27.091618 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad883bb-4f72-4a15-818f-7ecf82b702b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:27.093925 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad883bb-4f72-4a15-818f-7ecf82b702b3 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:27.115553 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:27.128363 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 995/2008] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:57:27 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:57:27.129764 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b9cd34-54f5-488d-8875-57d364948591 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:27.141763 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b9cd34-54f5-488d-8875-57d364948591 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:27.152820 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad883bb-4f72-4a15-818f-7ecf82b702b3 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:27.154896 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-bad883bb-4f72-4a15-818f-7ecf82b702b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:27.172582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:27.228885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad883bb-4f72-4a15-818f-7ecf82b702b3 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:27.229592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe995c1-abb0-47aa-aa4a-574aa8f6daaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:27.230629 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe995c1-abb0-47aa-aa4a-574aa8f6daaf admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:27.230989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe995c1-abb0-47aa-aa4a-574aa8f6daaf admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:27.231360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe995c1-abb0-47aa-aa4a-574aa8f6daaf admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:27.232144 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe995c1-abb0-47aa-aa4a-574aa8f6daaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:27.234557 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5cea1-ac55-4a14-94be-d80b547af003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:27.235047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe995c1-abb0-47aa-aa4a-574aa8f6daaf admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:27.235427 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5cea1-ac55-4a14-94be-d80b547af003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:27.235713 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5cea1-ac55-4a14-94be-d80b547af003 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:27.235991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5cea1-ac55-4a14-94be-d80b547af003 admin admin] PATH_INFO: `/v3/projects/2b000017a4144a188bbf1649b73df0a2` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:27.237209 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5cea1-ac55-4a14-94be-d80b547af003 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b000017a4144a188bbf1649b73df0a2)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:27.239285 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5cea1-ac55-4a14-94be-d80b547af003 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:27.261949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1014/2009] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:57:27 2022] POST /identity/v3/users => generated 288 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:57:27.263077 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe995c1-abb0-47aa-aa4a-574aa8f6daaf admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:27.273946 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe995c1-abb0-47aa-aa4a-574aa8f6daaf admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:27.284617 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f97407-4af2-4bd8-aed3-baedb352e559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:27.285427 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f97407-4af2-4bd8-aed3-baedb352e559 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:27.285956 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f97407-4af2-4bd8-aed3-baedb352e559 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:27.287954 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f97407-4af2-4bd8-aed3-baedb352e559 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:27.288714 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f97407-4af2-4bd8-aed3-baedb352e559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:27.290398 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f97407-4af2-4bd8-aed3-baedb352e559 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:27.302084 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1015/2010] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:57:27.305851 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f97407-4af2-4bd8-aed3-baedb352e559 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:27.306714 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-a8f5cea1-ac55-4a14-94be-d80b547af003 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b000017a4144a188bbf1649b73df0a2', 'request_id': 'req-a8f5cea1-ac55-4a14-94be-d80b547af003'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:27.312777 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-a8f5cea1-ac55-4a14-94be-d80b547af003 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b000017a4144a188bbf1649b73df0a2', 'request_id': 'req-a8f5cea1-ac55-4a14-94be-d80b547af003'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:27.316995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f97407-4af2-4bd8-aed3-baedb352e559 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:27.319717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-a8f5cea1-ac55-4a14-94be-d80b547af003 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b000017a4144a188bbf1649b73df0a2', 'request_id': 'req-a8f5cea1-ac55-4a14-94be-d80b547af003'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:27.333987 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-a8f5cea1-ac55-4a14-94be-d80b547af003 admin admin] The token cache is being invalidate because project 2b000017a4144a188bbf1649b73df0a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:57:27.334204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-a8f5cea1-ac55-4a14-94be-d80b547af003 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:27.334760 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-a8f5cea1-ac55-4a14-94be-d80b547af003 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:27.336349 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 996/2011] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:57:27 2022] DELETE /identity/v3/projects/2b000017a4144a188bbf1649b73df0a2 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:57:27.341141 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5cea1-ac55-4a14-94be-d80b547af003 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:27.342631 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-a8f5cea1-ac55-4a14-94be-d80b547af003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:27.399948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f8312-083b-48f6-a4d4-438ade6582b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:27.400700 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f8312-083b-48f6-a4d4-438ade6582b1 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:27.401057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f8312-083b-48f6-a4d4-438ade6582b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:27.401752 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f8312-083b-48f6-a4d4-438ade6582b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:27.402700 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f8312-083b-48f6-a4d4-438ade6582b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:27.404607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f8312-083b-48f6-a4d4-438ade6582b1 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:27.419651 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5cea1-ac55-4a14-94be-d80b547af003 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:27.420640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1016/2012] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:57:27.421611 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f8312-083b-48f6-a4d4-438ade6582b1 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:27.423469 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-511f8312-083b-48f6-a4d4-438ade6582b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:27.425857 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c6462-ded3-430d-a2ae-aa5094b6aff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:27.427598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c6462-ded3-430d-a2ae-aa5094b6aff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:27.427598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c6462-ded3-430d-a2ae-aa5094b6aff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:27.427598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c6462-ded3-430d-a2ae-aa5094b6aff0 admin admin] PATH_INFO: `/v3/users/9f5601f539fc429984047e6b7268ae01` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:27.428667 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70c6462-ded3-430d-a2ae-aa5094b6aff0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f5601f539fc429984047e6b7268ae01)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:27.431108 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70c6462-ded3-430d-a2ae-aa5094b6aff0 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:27.438543 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f8312-083b-48f6-a4d4-438ade6582b1 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:27.443827 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a3d93-270f-4167-b632-f1b704014917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:27.444999 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a3d93-270f-4167-b632-f1b704014917 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:27.445412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a3d93-270f-4167-b632-f1b704014917 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:27.445697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a3d93-270f-4167-b632-f1b704014917 admin admin] PATH_INFO: `/v3/projects/30e183b982804c7b858ff95dd533decf/users/cd9c8b056d80474b95b14f04ccc6c407/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:27.464270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a3d93-270f-4167-b632-f1b704014917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30e183b982804c7b858ff95dd533decf, user_id=cd9c8b056d80474b95b14f04ccc6c407, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:27.467379 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a3d93-270f-4167-b632-f1b704014917 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:27.472428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:27.491145 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1017/2013] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:57:27 2022] PUT /identity/v3/projects/30e183b982804c7b858ff95dd533decf/users/cd9c8b056d80474b95b14f04ccc6c407/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:27.496640 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b70c6462-ded3-430d-a2ae-aa5094b6aff0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f5601f539fc429984047e6b7268ae01'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:27.503807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c129ff-f21a-49d2-9c6d-1bdff37bbef9 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:27.503807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c129ff-f21a-49d2-9c6d-1bdff37bbef9 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:27.503807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c129ff-f21a-49d2-9c6d-1bdff37bbef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:27.512318 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b70c6462-ded3-430d-a2ae-aa5094b6aff0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f5601f539fc429984047e6b7268ae01'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:27.520849 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b70c6462-ded3-430d-a2ae-aa5094b6aff0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f5601f539fc429984047e6b7268ae01'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:27.521962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b70c6462-ded3-430d-a2ae-aa5094b6aff0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f5601f539fc429984047e6b7268ae01'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:27.522830 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b70c6462-ded3-430d-a2ae-aa5094b6aff0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f5601f539fc429984047e6b7268ae01'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:27.535655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-86c129ff-f21a-49d2-9c6d-1bdff37bbef9 None None] MFA Rules not processed for user `cd9c8b056d80474b95b14f04ccc6c407`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:27.537662 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-86c129ff-f21a-49d2-9c6d-1bdff37bbef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:27.543776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 997/2014] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:57:27 2022] DELETE /identity/v3/users/9f5601f539fc429984047e6b7268ae01 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:57:27.664461 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:27.670151 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1018/2015] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:57:27.682441 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c6462-ded3-430d-a2ae-aa5094b6aff0 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:27.683862 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-b70c6462-ded3-430d-a2ae-aa5094b6aff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:27.777267 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c6462-ded3-430d-a2ae-aa5094b6aff0 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:27.794894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e30635-0d25-45ac-ba17-bffd5a2e968b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88cbd76f70d447c4ba92777b2c09821d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:27.795521 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e30635-0d25-45ac-ba17-bffd5a2e968b service glance] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:27.795768 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e30635-0d25-45ac-ba17-bffd5a2e968b service glance] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:27.796040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e30635-0d25-45ac-ba17-bffd5a2e968b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:27.881377 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e30635-0d25-45ac-ba17-bffd5a2e968b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:27.883754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e30635-0d25-45ac-ba17-bffd5a2e968b service glance] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:27.913573 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 998/2016] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:57:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:57:28.230533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c129ff-f21a-49d2-9c6d-1bdff37bbef9 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:28.232917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-86c129ff-f21a-49d2-9c6d-1bdff37bbef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:28.365803 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c129ff-f21a-49d2-9c6d-1bdff37bbef9 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:28.457689 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aafd3f-c7b5-4e17-a55d-aaf842a86c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:28.458493 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aafd3f-c7b5-4e17-a55d-aaf842a86c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:28.458719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aafd3f-c7b5-4e17-a55d-aaf842a86c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:28.458998 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aafd3f-c7b5-4e17-a55d-aaf842a86c1e admin admin] PATH_INFO: `/v3/projects/901b7bea54a54497bc3cf53e7430d4ea` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:28.460588 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aafd3f-c7b5-4e17-a55d-aaf842a86c1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=901b7bea54a54497bc3cf53e7430d4ea)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:28.462715 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aafd3f-c7b5-4e17-a55d-aaf842a86c1e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:28.529551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-11aafd3f-c7b5-4e17-a55d-aaf842a86c1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '901b7bea54a54497bc3cf53e7430d4ea', 'request_id': 'req-11aafd3f-c7b5-4e17-a55d-aaf842a86c1e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:28.538587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-11aafd3f-c7b5-4e17-a55d-aaf842a86c1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '901b7bea54a54497bc3cf53e7430d4ea', 'request_id': 'req-11aafd3f-c7b5-4e17-a55d-aaf842a86c1e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:28.547034 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-11aafd3f-c7b5-4e17-a55d-aaf842a86c1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '901b7bea54a54497bc3cf53e7430d4ea', 'request_id': 'req-11aafd3f-c7b5-4e17-a55d-aaf842a86c1e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:28.569938 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-11aafd3f-c7b5-4e17-a55d-aaf842a86c1e admin admin] The token cache is being invalidate because project 901b7bea54a54497bc3cf53e7430d4ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:57:28.570301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-11aafd3f-c7b5-4e17-a55d-aaf842a86c1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:28.570971 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-11aafd3f-c7b5-4e17-a55d-aaf842a86c1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:28.573161 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1019/2017] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:57:28 2022] DELETE /identity/v3/projects/901b7bea54a54497bc3cf53e7430d4ea => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:28.577979 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8771190-49e3-44cb-8356-7fc319255dd4 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:28.578388 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8771190-49e3-44cb-8356-7fc319255dd4 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:28.578563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8771190-49e3-44cb-8356-7fc319255dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:28.603471 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-a8771190-49e3-44cb-8356-7fc319255dd4 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:28.606432 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-a8771190-49e3-44cb-8356-7fc319255dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:28.747388 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 999/2018] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:28.756165 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a26be-d070-416e-a56f-b22ae15c4de0 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:28.756438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a26be-d070-416e-a56f-b22ae15c4de0 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:28.756716 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a26be-d070-416e-a56f-b22ae15c4de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:28.777680 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-6a7a26be-d070-416e-a56f-b22ae15c4de0 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:28.779003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-6a7a26be-d070-416e-a56f-b22ae15c4de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:28.798605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1020/2019] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:28.802459 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8771190-49e3-44cb-8356-7fc319255dd4 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:28.807744 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:28.816957 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8771190-49e3-44cb-8356-7fc319255dd4 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:28.826271 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5090d-4137-4664-b92c-26204ca34edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:28.826944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5090d-4137-4664-b92c-26204ca34edf admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:28.827219 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5090d-4137-4664-b92c-26204ca34edf admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:28.827487 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5090d-4137-4664-b92c-26204ca34edf admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:28.828215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5090d-4137-4664-b92c-26204ca34edf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:57:28.828517 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5090d-4137-4664-b92c-26204ca34edf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:28.830343 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5090d-4137-4664-b92c-26204ca34edf admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:28.846045 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1000/2020] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:57:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:57:28.848520 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a26be-d070-416e-a56f-b22ae15c4de0 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:28.859857 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a26be-d070-416e-a56f-b22ae15c4de0 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:28.870154 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bace41-4f61-4914-831b-e4af81256211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:28.870993 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bace41-4f61-4914-831b-e4af81256211 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:28.871407 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bace41-4f61-4914-831b-e4af81256211 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:28.871753 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bace41-4f61-4914-831b-e4af81256211 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:28.872576 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bace41-4f61-4914-831b-e4af81256211 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:28.874579 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bace41-4f61-4914-831b-e4af81256211 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:28.909212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1021/2021] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:57:28 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:57:28.912820 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5090d-4137-4664-b92c-26204ca34edf admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:28.924532 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5090d-4137-4664-b92c-26204ca34edf admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:28.994047 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f767c-e2c2-41f9-b3fd-0a99127fe468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:28.994746 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f767c-e2c2-41f9-b3fd-0a99127fe468 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:28.995682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f767c-e2c2-41f9-b3fd-0a99127fe468 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:28.995911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f767c-e2c2-41f9-b3fd-0a99127fe468 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:28.996512 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f767c-e2c2-41f9-b3fd-0a99127fe468 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:29.010257 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f767c-e2c2-41f9-b3fd-0a99127fe468 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:29.031018 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1001/2022] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:57:28 2022] POST /identity/v3/users => generated 289 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:57:29.034937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bace41-4f61-4914-831b-e4af81256211 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:29.045844 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bace41-4f61-4914-831b-e4af81256211 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:29.050659 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ee0bd-8e26-430c-8d75-e27b61be203f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:29.051321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ee0bd-8e26-430c-8d75-e27b61be203f admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:29.051638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ee0bd-8e26-430c-8d75-e27b61be203f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:29.051924 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ee0bd-8e26-430c-8d75-e27b61be203f admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:29.052646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ee0bd-8e26-430c-8d75-e27b61be203f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:29.054287 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ee0bd-8e26-430c-8d75-e27b61be203f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:29.056382 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:29.066286 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1022/2023] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:57:29.069870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f767c-e2c2-41f9-b3fd-0a99127fe468 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:29.079865 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f767c-e2c2-41f9-b3fd-0a99127fe468 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:29.084462 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2644fa9-8cd4-4e2b-b29f-c21973201e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:29.085115 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2644fa9-8cd4-4e2b-b29f-c21973201e98 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:29.085398 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2644fa9-8cd4-4e2b-b29f-c21973201e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:29.085674 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2644fa9-8cd4-4e2b-b29f-c21973201e98 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:29.086430 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2644fa9-8cd4-4e2b-b29f-c21973201e98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:29.088023 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2644fa9-8cd4-4e2b-b29f-c21973201e98 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:29.100180 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1002/2024] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:57:29.104931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ee0bd-8e26-430c-8d75-e27b61be203f admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:29.117134 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ee0bd-8e26-430c-8d75-e27b61be203f admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:29.122397 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e2098-edd7-4908-8c49-79edf14be76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:29.123310 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e2098-edd7-4908-8c49-79edf14be76a admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:29.123592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e2098-edd7-4908-8c49-79edf14be76a admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:29.123871 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e2098-edd7-4908-8c49-79edf14be76a admin admin] PATH_INFO: `/v3/projects/597727823ef34e0fa21c7e74148e30eb/users/2753bb5222c4432eb10757771b2020fe/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:29.139395 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e2098-edd7-4908-8c49-79edf14be76a admin admin] RBAC: Authorizing `identity:create_grant(project_id=597727823ef34e0fa21c7e74148e30eb, user_id=2753bb5222c4432eb10757771b2020fe, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:29.142122 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e2098-edd7-4908-8c49-79edf14be76a admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:29.156145 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1023/2025] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:57:29 2022] PUT /identity/v3/projects/597727823ef34e0fa21c7e74148e30eb/users/2753bb5222c4432eb10757771b2020fe/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:29.166507 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f8cc7-df76-4e5a-bbe2-e8b816f10b0e None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:29.166858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f8cc7-df76-4e5a-bbe2-e8b816f10b0e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:29.167169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f8cc7-df76-4e5a-bbe2-e8b816f10b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:29.195152 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-d39f8cc7-df76-4e5a-bbe2-e8b816f10b0e None None] MFA Rules not processed for user `2753bb5222c4432eb10757771b2020fe`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:29.196370 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d39f8cc7-df76-4e5a-bbe2-e8b816f10b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:29.265124 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1003/2026] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:29.305933 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e2098-edd7-4908-8c49-79edf14be76a admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:29.308318 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-240e2098-edd7-4908-8c49-79edf14be76a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:29.373455 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e2098-edd7-4908-8c49-79edf14be76a admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:29.377293 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2203e-53ef-47b5-8f63-fba225abdf25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:29.377954 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2203e-53ef-47b5-8f63-fba225abdf25 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:29.378268 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2203e-53ef-47b5-8f63-fba225abdf25 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:29.378544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2203e-53ef-47b5-8f63-fba225abdf25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:29.387494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2203e-53ef-47b5-8f63-fba225abdf25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:29.388939 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2203e-53ef-47b5-8f63-fba225abdf25 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:29.398289 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1024/2027] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:57:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:57:39.262072 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=71,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:39.394286 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=37,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:45.173713 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f8cc7-df76-4e5a-bbe2-e8b816f10b0e None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:45.175316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d39f8cc7-df76-4e5a-bbe2-e8b816f10b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:45.244340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f8cc7-df76-4e5a-bbe2-e8b816f10b0e None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:45.249025 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297a807-bc88-46fa-a3ed-b3c1d49f0a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:45.249934 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297a807-bc88-46fa-a3ed-b3c1d49f0a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:45.250331 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297a807-bc88-46fa-a3ed-b3c1d49f0a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:45.250513 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297a807-bc88-46fa-a3ed-b3c1d49f0a41 admin admin] PATH_INFO: `/v3/users/b3cd4cec00fe4f0a8dafba7dc16d685e` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:45.251742 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6297a807-bc88-46fa-a3ed-b3c1d49f0a41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3cd4cec00fe4f0a8dafba7dc16d685e)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:45.253793 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6297a807-bc88-46fa-a3ed-b3c1d49f0a41 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:45.302816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6297a807-bc88-46fa-a3ed-b3c1d49f0a41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3cd4cec00fe4f0a8dafba7dc16d685e'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:45.318710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6297a807-bc88-46fa-a3ed-b3c1d49f0a41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3cd4cec00fe4f0a8dafba7dc16d685e'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:45.326900 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6297a807-bc88-46fa-a3ed-b3c1d49f0a41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3cd4cec00fe4f0a8dafba7dc16d685e'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:45.327742 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6297a807-bc88-46fa-a3ed-b3c1d49f0a41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3cd4cec00fe4f0a8dafba7dc16d685e'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:45.328369 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6297a807-bc88-46fa-a3ed-b3c1d49f0a41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3cd4cec00fe4f0a8dafba7dc16d685e'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:45.342300 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1004/2028] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:57:45 2022] DELETE /identity/v3/users/b3cd4cec00fe4f0a8dafba7dc16d685e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:45.549918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2203e-53ef-47b5-8f63-fba225abdf25 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:45.582721 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2203e-53ef-47b5-8f63-fba225abdf25 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:45.658400 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9da28-8d8d-445c-8e27-d56bc38627a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:45.659373 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9da28-8d8d-445c-8e27-d56bc38627a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:45.659741 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9da28-8d8d-445c-8e27-d56bc38627a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:45.660068 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9da28-8d8d-445c-8e27-d56bc38627a9 admin admin] PATH_INFO: `/v3/projects/d3fab2e6427d486bb1744239b759c415` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:45.661563 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9da28-8d8d-445c-8e27-d56bc38627a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3fab2e6427d486bb1744239b759c415)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:45.663834 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9da28-8d8d-445c-8e27-d56bc38627a9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:45.715449 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-40f9da28-8d8d-445c-8e27-d56bc38627a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3fab2e6427d486bb1744239b759c415', 'request_id': 'req-40f9da28-8d8d-445c-8e27-d56bc38627a9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:45.721214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-40f9da28-8d8d-445c-8e27-d56bc38627a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3fab2e6427d486bb1744239b759c415', 'request_id': 'req-40f9da28-8d8d-445c-8e27-d56bc38627a9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:45.727696 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-40f9da28-8d8d-445c-8e27-d56bc38627a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3fab2e6427d486bb1744239b759c415', 'request_id': 'req-40f9da28-8d8d-445c-8e27-d56bc38627a9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:45.742152 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-40f9da28-8d8d-445c-8e27-d56bc38627a9 admin admin] The token cache is being invalidate because project d3fab2e6427d486bb1744239b759c415 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:57:45.742445 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-40f9da28-8d8d-445c-8e27-d56bc38627a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:45.743083 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-40f9da28-8d8d-445c-8e27-d56bc38627a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:45.744641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1025/2029] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:57:45 2022] DELETE /identity/v3/projects/d3fab2e6427d486bb1744239b759c415 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:45.751693 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2610bac-58e6-46b7-bc8a-c0fa5904c5bb None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:45.751988 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2610bac-58e6-46b7-bc8a-c0fa5904c5bb None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:45.752329 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2610bac-58e6-46b7-bc8a-c0fa5904c5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:45.784040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-a2610bac-58e6-46b7-bc8a-c0fa5904c5bb None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:45.785310 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-a2610bac-58e6-46b7-bc8a-c0fa5904c5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:45.845757 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1005/2030] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:45.853696 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a2f45-c62e-437b-a57a-b720c0e1bf94 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:45.853991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a2f45-c62e-437b-a57a-b720c0e1bf94 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:45.854306 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a2f45-c62e-437b-a57a-b720c0e1bf94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:45.872924 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-584a2f45-c62e-437b-a57a-b720c0e1bf94 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:45.874166 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-584a2f45-c62e-437b-a57a-b720c0e1bf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:45.882088 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1026/2031] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:45.886888 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2610bac-58e6-46b7-bc8a-c0fa5904c5bb None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:45.897291 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2610bac-58e6-46b7-bc8a-c0fa5904c5bb None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:45.902638 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b34394-8ef7-4438-9553-3b51185113c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:45.903302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b34394-8ef7-4438-9553-3b51185113c5 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:45.903586 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b34394-8ef7-4438-9553-3b51185113c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:45.903863 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b34394-8ef7-4438-9553-3b51185113c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:45.904599 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b34394-8ef7-4438-9553-3b51185113c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:57:45.904904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b34394-8ef7-4438-9553-3b51185113c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:45.906540 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b34394-8ef7-4438-9553-3b51185113c5 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:45.913641 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:45.920138 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1006/2032] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:57:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:57:45.924434 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a2f45-c62e-437b-a57a-b720c0e1bf94 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:45.933886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a2f45-c62e-437b-a57a-b720c0e1bf94 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:45.938722 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecba63-f75e-4e75-a94e-d09a2504e7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:45.939390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecba63-f75e-4e75-a94e-d09a2504e7eb admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:45.939677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecba63-f75e-4e75-a94e-d09a2504e7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:45.939943 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecba63-f75e-4e75-a94e-d09a2504e7eb admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:45.940643 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ecba63-f75e-4e75-a94e-d09a2504e7eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:45.942599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ecba63-f75e-4e75-a94e-d09a2504e7eb admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:45.970624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1027/2033] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:57:45 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:57:45.975066 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b34394-8ef7-4438-9553-3b51185113c5 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:45.988379 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b34394-8ef7-4438-9553-3b51185113c5 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:46.066077 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37862e3-f2bc-4214-8f1a-d045b89f1546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:46.066838 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37862e3-f2bc-4214-8f1a-d045b89f1546 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:46.067020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37862e3-f2bc-4214-8f1a-d045b89f1546 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:46.067327 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37862e3-f2bc-4214-8f1a-d045b89f1546 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:46.068296 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37862e3-f2bc-4214-8f1a-d045b89f1546 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:46.069906 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37862e3-f2bc-4214-8f1a-d045b89f1546 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:46.098438 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1007/2034] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:57:45 2022] POST /identity/v3/users => generated 293 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:57:46.102960 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecba63-f75e-4e75-a94e-d09a2504e7eb admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:46.113442 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecba63-f75e-4e75-a94e-d09a2504e7eb admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:46.118024 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa22a52-edb7-4974-9166-b196e1d3aa9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:46.118692 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa22a52-edb7-4974-9166-b196e1d3aa9d admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:46.119516 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa22a52-edb7-4974-9166-b196e1d3aa9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:46.119790 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa22a52-edb7-4974-9166-b196e1d3aa9d admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:46.120510 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa22a52-edb7-4974-9166-b196e1d3aa9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:46.122124 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa22a52-edb7-4974-9166-b196e1d3aa9d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:46.138160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1028/2035] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:57:46.138951 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37862e3-f2bc-4214-8f1a-d045b89f1546 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:46.150364 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37862e3-f2bc-4214-8f1a-d045b89f1546 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:46.156190 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b31aa-9980-4567-a29f-ea0834c8853d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:46.157445 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b31aa-9980-4567-a29f-ea0834c8853d admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:46.157866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b31aa-9980-4567-a29f-ea0834c8853d admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:46.158228 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b31aa-9980-4567-a29f-ea0834c8853d admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:46.159082 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b31aa-9980-4567-a29f-ea0834c8853d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:46.160672 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b31aa-9980-4567-a29f-ea0834c8853d admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:46.181000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1008/2036] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:57:46.183681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa22a52-edb7-4974-9166-b196e1d3aa9d admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:46.200335 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa22a52-edb7-4974-9166-b196e1d3aa9d admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:46.205320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ecbdcf-7b25-49dc-b720-f7796ac8589b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:46.206252 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ecbdcf-7b25-49dc-b720-f7796ac8589b admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:46.206553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ecbdcf-7b25-49dc-b720-f7796ac8589b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:46.206807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ecbdcf-7b25-49dc-b720-f7796ac8589b admin admin] PATH_INFO: `/v3/projects/16edf1463a93425298357fc12752c5a9/users/fda1d04cf14641a2a6ed2152430ae845/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:46.223453 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ecbdcf-7b25-49dc-b720-f7796ac8589b admin admin] RBAC: Authorizing `identity:create_grant(project_id=16edf1463a93425298357fc12752c5a9, user_id=fda1d04cf14641a2a6ed2152430ae845, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:46.226208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ecbdcf-7b25-49dc-b720-f7796ac8589b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:46.240823 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1029/2037] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:57:46 2022] PUT /identity/v3/projects/16edf1463a93425298357fc12752c5a9/users/fda1d04cf14641a2a6ed2152430ae845/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:57:46.252683 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a9756-f6db-4761-ac4b-ebeedeb3c4df None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:46.253027 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a9756-f6db-4761-ac4b-ebeedeb3c4df None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:46.253303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a9756-f6db-4761-ac4b-ebeedeb3c4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:46.287137 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-272a9756-f6db-4761-ac4b-ebeedeb3c4df None None] MFA Rules not processed for user `fda1d04cf14641a2a6ed2152430ae845`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:46.289189 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-272a9756-f6db-4761-ac4b-ebeedeb3c4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:46.359826 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1009/2038] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:57:46.373496 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ecbdcf-7b25-49dc-b720-f7796ac8589b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:46.378659 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-25ecbdcf-7b25-49dc-b720-f7796ac8589b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:46.412237 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:46.490673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ecbdcf-7b25-49dc-b720-f7796ac8589b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:46.497731 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853eec1-2c1d-475d-a45e-05e90176bad0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:46.498543 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853eec1-2c1d-475d-a45e-05e90176bad0 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:46.498838 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853eec1-2c1d-475d-a45e-05e90176bad0 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:46.499120 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853eec1-2c1d-475d-a45e-05e90176bad0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:46.507605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853eec1-2c1d-475d-a45e-05e90176bad0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:46.509396 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853eec1-2c1d-475d-a45e-05e90176bad0 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:46.519075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1030/2039] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:57:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:57:48.100514 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a9756-f6db-4761-ac4b-ebeedeb3c4df None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:48.102034 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-272a9756-f6db-4761-ac4b-ebeedeb3c4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:48.174394 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a9756-f6db-4761-ac4b-ebeedeb3c4df None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:48.181523 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c73c7-a864-4600-a4c0-ec7c4741dcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:48.182577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c73c7-a864-4600-a4c0-ec7c4741dcd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:48.183020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c73c7-a864-4600-a4c0-ec7c4741dcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:48.183144 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c73c7-a864-4600-a4c0-ec7c4741dcd8 admin admin] PATH_INFO: `/v3/users/b140f43a113849869e5f7bf9b55bc8fa` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:48.184496 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c73c7-a864-4600-a4c0-ec7c4741dcd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b140f43a113849869e5f7bf9b55bc8fa)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:48.186610 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c73c7-a864-4600-a4c0-ec7c4741dcd8 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:48.192790 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:48.246323 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-102c73c7-a864-4600-a4c0-ec7c4741dcd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b140f43a113849869e5f7bf9b55bc8fa'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:48.265260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-102c73c7-a864-4600-a4c0-ec7c4741dcd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b140f43a113849869e5f7bf9b55bc8fa'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:48.283704 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-102c73c7-a864-4600-a4c0-ec7c4741dcd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b140f43a113849869e5f7bf9b55bc8fa'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:48.285571 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-102c73c7-a864-4600-a4c0-ec7c4741dcd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b140f43a113849869e5f7bf9b55bc8fa'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:48.288225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-102c73c7-a864-4600-a4c0-ec7c4741dcd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b140f43a113849869e5f7bf9b55bc8fa'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:48.306809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1010/2040] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:57:48 2022] DELETE /identity/v3/users/b140f43a113849869e5f7bf9b55bc8fa => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:48.312786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362406ed-1711-46e6-bae7-74a65c5b52cb None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:48.312786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362406ed-1711-46e6-bae7-74a65c5b52cb None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:48.312786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362406ed-1711-46e6-bae7-74a65c5b52cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:48.344948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-362406ed-1711-46e6-bae7-74a65c5b52cb None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:48.346236 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-362406ed-1711-46e6-bae7-74a65c5b52cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:48.422810 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1031/2041] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:48.435326 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c73c7-a864-4600-a4c0-ec7c4741dcd8 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:48.436940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-102c73c7-a864-4600-a4c0-ec7c4741dcd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:48.499965 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c73c7-a864-4600-a4c0-ec7c4741dcd8 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:48.518942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda15a15-3b8b-4df5-8d7d-5f4e3744d69e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:48.519718 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda15a15-3b8b-4df5-8d7d-5f4e3744d69e service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:48.520076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda15a15-3b8b-4df5-8d7d-5f4e3744d69e service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:48.520448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda15a15-3b8b-4df5-8d7d-5f4e3744d69e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:48.530610 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda15a15-3b8b-4df5-8d7d-5f4e3744d69e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:48.532117 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda15a15-3b8b-4df5-8d7d-5f4e3744d69e service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:48.542811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1011/2042] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:57:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:57:48.932475 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362406ed-1711-46e6-bae7-74a65c5b52cb None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:48.933902 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-362406ed-1711-46e6-bae7-74a65c5b52cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:48.946420 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362406ed-1711-46e6-bae7-74a65c5b52cb None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:48.951102 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652acf73-e5db-4a99-868c-19d1fdcd6d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:48.951967 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652acf73-e5db-4a99-868c-19d1fdcd6d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:48.952250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652acf73-e5db-4a99-868c-19d1fdcd6d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:48.952539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652acf73-e5db-4a99-868c-19d1fdcd6d1f admin admin] PATH_INFO: `/v3/projects/66d1682b757348df9fb1240bf9a7edd9` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:48.953724 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652acf73-e5db-4a99-868c-19d1fdcd6d1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=66d1682b757348df9fb1240bf9a7edd9)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:48.955687 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652acf73-e5db-4a99-868c-19d1fdcd6d1f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:49.004155 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-652acf73-e5db-4a99-868c-19d1fdcd6d1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66d1682b757348df9fb1240bf9a7edd9', 'request_id': 'req-652acf73-e5db-4a99-868c-19d1fdcd6d1f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:49.010399 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-652acf73-e5db-4a99-868c-19d1fdcd6d1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66d1682b757348df9fb1240bf9a7edd9', 'request_id': 'req-652acf73-e5db-4a99-868c-19d1fdcd6d1f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:49.016678 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-652acf73-e5db-4a99-868c-19d1fdcd6d1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66d1682b757348df9fb1240bf9a7edd9', 'request_id': 'req-652acf73-e5db-4a99-868c-19d1fdcd6d1f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:49.029507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-652acf73-e5db-4a99-868c-19d1fdcd6d1f admin admin] The token cache is being invalidate because project 66d1682b757348df9fb1240bf9a7edd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:57:49.029800 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-652acf73-e5db-4a99-868c-19d1fdcd6d1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:49.030461 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-652acf73-e5db-4a99-868c-19d1fdcd6d1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:49.032001 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1032/2043] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:57:48 2022] DELETE /identity/v3/projects/66d1682b757348df9fb1240bf9a7edd9 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:49.036185 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda15a15-3b8b-4df5-8d7d-5f4e3744d69e service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:49.046032 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda15a15-3b8b-4df5-8d7d-5f4e3744d69e service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:49.108908 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a933c1d-c3c0-47b9-ae4b-948ffd3a643a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:49.109789 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a933c1d-c3c0-47b9-ae4b-948ffd3a643a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:49.110105 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a933c1d-c3c0-47b9-ae4b-948ffd3a643a admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:49.110382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a933c1d-c3c0-47b9-ae4b-948ffd3a643a admin admin] PATH_INFO: `/v3/users/41216984dda84a9c90c7a3db28a0bac1` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:49.111589 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a933c1d-c3c0-47b9-ae4b-948ffd3a643a admin admin] RBAC: Authorizing `identity:delete_user(user_id=41216984dda84a9c90c7a3db28a0bac1)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:49.113525 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a933c1d-c3c0-47b9-ae4b-948ffd3a643a admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:49.149489 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:49.164190 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3a933c1d-c3c0-47b9-ae4b-948ffd3a643a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41216984dda84a9c90c7a3db28a0bac1'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:49.176511 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3a933c1d-c3c0-47b9-ae4b-948ffd3a643a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41216984dda84a9c90c7a3db28a0bac1'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:49.183942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3a933c1d-c3c0-47b9-ae4b-948ffd3a643a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41216984dda84a9c90c7a3db28a0bac1'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:49.184598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3a933c1d-c3c0-47b9-ae4b-948ffd3a643a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41216984dda84a9c90c7a3db28a0bac1'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:49.185199 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3a933c1d-c3c0-47b9-ae4b-948ffd3a643a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41216984dda84a9c90c7a3db28a0bac1'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:49.197568 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1012/2044] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:57:49 2022] DELETE /identity/v3/users/41216984dda84a9c90c7a3db28a0bac1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:49.486471 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652acf73-e5db-4a99-868c-19d1fdcd6d1f admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:49.487923 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-652acf73-e5db-4a99-868c-19d1fdcd6d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:49.502970 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652acf73-e5db-4a99-868c-19d1fdcd6d1f admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:49.508197 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:49.568863 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7661a-cb84-4bf2-88f1-147a1b867092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:49.569740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7661a-cb84-4bf2-88f1-147a1b867092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:49.570028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7661a-cb84-4bf2-88f1-147a1b867092 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:49.570338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7661a-cb84-4bf2-88f1-147a1b867092 admin admin] PATH_INFO: `/v3/projects/175e1034a7ae4f7a821046b6b678e55a` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:49.571532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f7661a-cb84-4bf2-88f1-147a1b867092 admin admin] RBAC: Authorizing `identity:delete_project(project_id=175e1034a7ae4f7a821046b6b678e55a)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:49.574024 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f7661a-cb84-4bf2-88f1-147a1b867092 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:49.623021 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-39f7661a-cb84-4bf2-88f1-147a1b867092 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '175e1034a7ae4f7a821046b6b678e55a', 'request_id': 'req-39f7661a-cb84-4bf2-88f1-147a1b867092'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:49.628494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-39f7661a-cb84-4bf2-88f1-147a1b867092 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '175e1034a7ae4f7a821046b6b678e55a', 'request_id': 'req-39f7661a-cb84-4bf2-88f1-147a1b867092'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:49.634860 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-39f7661a-cb84-4bf2-88f1-147a1b867092 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '175e1034a7ae4f7a821046b6b678e55a', 'request_id': 'req-39f7661a-cb84-4bf2-88f1-147a1b867092'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:49.647616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-39f7661a-cb84-4bf2-88f1-147a1b867092 admin admin] The token cache is being invalidate because project 175e1034a7ae4f7a821046b6b678e55a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:57:49.647909 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-39f7661a-cb84-4bf2-88f1-147a1b867092 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:49.648537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-39f7661a-cb84-4bf2-88f1-147a1b867092 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:49.650107 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1033/2045] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:57:49 2022] DELETE /identity/v3/projects/175e1034a7ae4f7a821046b6b678e55a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:49.655130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cf6dc-216e-47b4-839b-bf8d55a6322e None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:49.655388 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cf6dc-216e-47b4-839b-bf8d55a6322e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:49.655695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cf6dc-216e-47b4-839b-bf8d55a6322e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:49.674646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-5c1cf6dc-216e-47b4-839b-bf8d55a6322e None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:49.675839 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-5c1cf6dc-216e-47b4-839b-bf8d55a6322e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:49.740232 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1013/2046] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:49.751306 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426a68b-bb1b-46d0-b081-00bd1454af6f None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:49.751601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426a68b-bb1b-46d0-b081-00bd1454af6f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:49.751885 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426a68b-bb1b-46d0-b081-00bd1454af6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:49.773836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-2426a68b-bb1b-46d0-b081-00bd1454af6f None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:49.776027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2426a68b-bb1b-46d0-b081-00bd1454af6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:49.787342 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1034/2047] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:49.792248 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cf6dc-216e-47b4-839b-bf8d55a6322e None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:49.802292 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cf6dc-216e-47b4-839b-bf8d55a6322e None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:49.810851 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586aaa62-3e0e-49c1-bcf4-5b857dc3843a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:49.812048 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586aaa62-3e0e-49c1-bcf4-5b857dc3843a admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:49.812549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586aaa62-3e0e-49c1-bcf4-5b857dc3843a admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:49.813042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586aaa62-3e0e-49c1-bcf4-5b857dc3843a admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:49.814323 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586aaa62-3e0e-49c1-bcf4-5b857dc3843a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:57:49.815133 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586aaa62-3e0e-49c1-bcf4-5b857dc3843a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:49.816892 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586aaa62-3e0e-49c1-bcf4-5b857dc3843a admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:49.829784 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1014/2048] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:57:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:57:49.834329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426a68b-bb1b-46d0-b081-00bd1454af6f None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:49.844390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426a68b-bb1b-46d0-b081-00bd1454af6f None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:49.849547 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f370d-d4c9-48a5-b44e-b0c1228a7148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:49.850244 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f370d-d4c9-48a5-b44e-b0c1228a7148 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:49.850536 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f370d-d4c9-48a5-b44e-b0c1228a7148 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:49.850817 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f370d-d4c9-48a5-b44e-b0c1228a7148 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:49.851528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f370d-d4c9-48a5-b44e-b0c1228a7148 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:49.854514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f370d-d4c9-48a5-b44e-b0c1228a7148 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:49.882489 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1035/2049] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:57:49 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:57:49.887848 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586aaa62-3e0e-49c1-bcf4-5b857dc3843a admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:49.899449 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586aaa62-3e0e-49c1-bcf4-5b857dc3843a admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:49.967737 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92910752-469a-4bd0-b0da-8266f4b7a20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:49.968419 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92910752-469a-4bd0-b0da-8266f4b7a20b admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:49.968700 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92910752-469a-4bd0-b0da-8266f4b7a20b admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:49.968974 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92910752-469a-4bd0-b0da-8266f4b7a20b admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:49.969683 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92910752-469a-4bd0-b0da-8266f4b7a20b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:49.971700 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92910752-469a-4bd0-b0da-8266f4b7a20b admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:49.993359 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1015/2050] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:57:49 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:57:49.997416 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f370d-d4c9-48a5-b44e-b0c1228a7148 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:50.008327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f370d-d4c9-48a5-b44e-b0c1228a7148 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:50.013033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f231e0-8a60-4551-a837-f598d7c7ab8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:50.013701 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f231e0-8a60-4551-a837-f598d7c7ab8f admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:50.013990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f231e0-8a60-4551-a837-f598d7c7ab8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:50.014300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f231e0-8a60-4551-a837-f598d7c7ab8f admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:50.015026 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f231e0-8a60-4551-a837-f598d7c7ab8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:50.016643 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f231e0-8a60-4551-a837-f598d7c7ab8f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:50.028818 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1036/2051] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:57:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:57:50.032977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92910752-469a-4bd0-b0da-8266f4b7a20b admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:50.043828 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92910752-469a-4bd0-b0da-8266f4b7a20b admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:50.048539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441e0437-cf89-4f46-8bb9-ef500977f9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:50.049119 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e0437-cf89-4f46-8bb9-ef500977f9cb admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:50.049395 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e0437-cf89-4f46-8bb9-ef500977f9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:50.049680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e0437-cf89-4f46-8bb9-ef500977f9cb admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:50.050452 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441e0437-cf89-4f46-8bb9-ef500977f9cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:50.052045 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441e0437-cf89-4f46-8bb9-ef500977f9cb admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:50.062657 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1016/2052] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:57:50.066843 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f231e0-8a60-4551-a837-f598d7c7ab8f admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:50.077643 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f231e0-8a60-4551-a837-f598d7c7ab8f admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:50.083099 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c285c0d-f940-4677-9991-69d2da173e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:50.084012 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c285c0d-f940-4677-9991-69d2da173e57 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:50.084294 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c285c0d-f940-4677-9991-69d2da173e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:50.084574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c285c0d-f940-4677-9991-69d2da173e57 admin admin] PATH_INFO: `/v3/projects/484e089945844e1c814e75f11ec9001d/users/87141436bd7745dba48f9eae60994149/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:50.107506 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c285c0d-f940-4677-9991-69d2da173e57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=484e089945844e1c814e75f11ec9001d, user_id=87141436bd7745dba48f9eae60994149, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:50.110157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c285c0d-f940-4677-9991-69d2da173e57 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:50.126815 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1037/2053] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:57:50 2022] PUT /identity/v3/projects/484e089945844e1c814e75f11ec9001d/users/87141436bd7745dba48f9eae60994149/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:50.141169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e30aaa-424a-41e3-9f4f-db7b3a1939f4 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:50.141706 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e30aaa-424a-41e3-9f4f-db7b3a1939f4 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:50.141706 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e30aaa-424a-41e3-9f4f-db7b3a1939f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:50.166958 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:50.172924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-d4e30aaa-424a-41e3-9f4f-db7b3a1939f4 None None] MFA Rules not processed for user `87141436bd7745dba48f9eae60994149`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:50.174985 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d4e30aaa-424a-41e3-9f4f-db7b3a1939f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:50.243165 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1017/2054] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:50.248008 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c285c0d-f940-4677-9991-69d2da173e57 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:50.287930 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:50.313450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c285c0d-f940-4677-9991-69d2da173e57 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:50.318112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b11183-664f-42fc-be3c-40a60a5cfe09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:50.318767 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b11183-664f-42fc-be3c-40a60a5cfe09 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:50.319045 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b11183-664f-42fc-be3c-40a60a5cfe09 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:50.319315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b11183-664f-42fc-be3c-40a60a5cfe09 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:50.320012 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b11183-664f-42fc-be3c-40a60a5cfe09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:50.322481 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b11183-664f-42fc-be3c-40a60a5cfe09 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:50.348050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1038/2055] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:57:50 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:57:50.352228 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e30aaa-424a-41e3-9f4f-db7b3a1939f4 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:50.363127 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e30aaa-424a-41e3-9f4f-db7b3a1939f4 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:50.424515 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01cfe63-2725-404f-b9a7-fba85e103210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:50.425955 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01cfe63-2725-404f-b9a7-fba85e103210 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:50.426449 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01cfe63-2725-404f-b9a7-fba85e103210 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:50.426901 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01cfe63-2725-404f-b9a7-fba85e103210 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:50.428096 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01cfe63-2725-404f-b9a7-fba85e103210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:50.432115 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01cfe63-2725-404f-b9a7-fba85e103210 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:50.470558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1018/2056] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:57:50 2022] POST /identity/v3/users => generated 296 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:57:50.479746 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b11183-664f-42fc-be3c-40a60a5cfe09 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:50.487408 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b11183-664f-42fc-be3c-40a60a5cfe09 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:50.493743 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54fb7a-9404-4f3b-bcca-0d1cdd12d8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:50.497088 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54fb7a-9404-4f3b-bcca-0d1cdd12d8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:50.497088 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54fb7a-9404-4f3b-bcca-0d1cdd12d8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:50.497088 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54fb7a-9404-4f3b-bcca-0d1cdd12d8e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:50.497088 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54fb7a-9404-4f3b-bcca-0d1cdd12d8e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:50.497484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54fb7a-9404-4f3b-bcca-0d1cdd12d8e6 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:50.509676 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1039/2057] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:57:50.513767 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01cfe63-2725-404f-b9a7-fba85e103210 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:50.524580 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01cfe63-2725-404f-b9a7-fba85e103210 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:50.530052 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be62c4-71a5-4b62-b305-033aed0f9add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:50.530959 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be62c4-71a5-4b62-b305-033aed0f9add admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:50.531243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be62c4-71a5-4b62-b305-033aed0f9add admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:50.531519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be62c4-71a5-4b62-b305-033aed0f9add admin admin] PATH_INFO: `/v3/projects/ed0fa65aba4b48a6a43e72bbc0bab060/users/2941dca04fea4633b22a0c503fddb6ec/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:50.548469 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be62c4-71a5-4b62-b305-033aed0f9add admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed0fa65aba4b48a6a43e72bbc0bab060, user_id=2941dca04fea4633b22a0c503fddb6ec, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:50.551201 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be62c4-71a5-4b62-b305-033aed0f9add admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:50.565734 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1019/2058] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:57:50 2022] PUT /identity/v3/projects/ed0fa65aba4b48a6a43e72bbc0bab060/users/2941dca04fea4633b22a0c503fddb6ec/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:50.570064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54fb7a-9404-4f3b-bcca-0d1cdd12d8e6 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:50.580472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54fb7a-9404-4f3b-bcca-0d1cdd12d8e6 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:50.639811 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e541d-5cc7-457c-878b-ca89948ee16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:50.640485 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e541d-5cc7-457c-878b-ca89948ee16f admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:50.640777 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e541d-5cc7-457c-878b-ca89948ee16f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:50.641053 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e541d-5cc7-457c-878b-ca89948ee16f admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:50.642078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e541d-5cc7-457c-878b-ca89948ee16f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:50.643693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e541d-5cc7-457c-878b-ca89948ee16f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:50.655141 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1040/2059] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:57:50.659728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be62c4-71a5-4b62-b305-033aed0f9add admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:50.670971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be62c4-71a5-4b62-b305-033aed0f9add admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:50.675704 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0e16d-826a-496a-a6ab-ff043dd52945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:50.676586 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0e16d-826a-496a-a6ab-ff043dd52945 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:50.676862 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0e16d-826a-496a-a6ab-ff043dd52945 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:50.677131 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0e16d-826a-496a-a6ab-ff043dd52945 admin admin] PATH_INFO: `/v3/projects/ed0fa65aba4b48a6a43e72bbc0bab060/users/2941dca04fea4633b22a0c503fddb6ec/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:50.679144 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd0e16d-826a-496a-a6ab-ff043dd52945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed0fa65aba4b48a6a43e72bbc0bab060, user_id=2941dca04fea4633b22a0c503fddb6ec, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:50.681833 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd0e16d-826a-496a-a6ab-ff043dd52945 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:50.692441 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1020/2060] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:57:50 2022] PUT /identity/v3/projects/ed0fa65aba4b48a6a43e72bbc0bab060/users/2941dca04fea4633b22a0c503fddb6ec/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:57:50.702527 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335cc8d3-1236-489d-bbc1-c47e7f209100 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:50.702754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335cc8d3-1236-489d-bbc1-c47e7f209100 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:50.703033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335cc8d3-1236-489d-bbc1-c47e7f209100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:50.730618 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-335cc8d3-1236-489d-bbc1-c47e7f209100 None None] MFA Rules not processed for user `2941dca04fea4633b22a0c503fddb6ec`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:50.731831 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-335cc8d3-1236-489d-bbc1-c47e7f209100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:50.811936 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1041/2061] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:50.823138 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0e16d-826a-496a-a6ab-ff043dd52945 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:50.826654 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-0dd0e16d-826a-496a-a6ab-ff043dd52945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:50.892381 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:50.914891 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0e16d-826a-496a-a6ab-ff043dd52945 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:50.918614 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f1c76-8ca9-43ff-8868-88d68f406e83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:50.919257 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f1c76-8ca9-43ff-8868-88d68f406e83 service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:50.919773 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f1c76-8ca9-43ff-8868-88d68f406e83 service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:50.919887 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f1c76-8ca9-43ff-8868-88d68f406e83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:50.928965 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f1c76-8ca9-43ff-8868-88d68f406e83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:50.930413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f1c76-8ca9-43ff-8868-88d68f406e83 service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:50.943335 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1021/2062] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:57:50 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:57:51.023747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335cc8d3-1236-489d-bbc1-c47e7f209100 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:51.034667 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335cc8d3-1236-489d-bbc1-c47e7f209100 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:51.038757 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0e1d5-2e79-40db-b016-aa5aed0bcbca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:51.039486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0e1d5-2e79-40db-b016-aa5aed0bcbca service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:51.039849 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0e1d5-2e79-40db-b016-aa5aed0bcbca service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:51.040194 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0e1d5-2e79-40db-b016-aa5aed0bcbca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:51.071099 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:51.105049 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee0e1d5-2e79-40db-b016-aa5aed0bcbca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:51.106622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee0e1d5-2e79-40db-b016-aa5aed0bcbca service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:51.115975 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1042/2063] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:57:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:57:58.340145 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f1c76-8ca9-43ff-8868-88d68f406e83 service cinder] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:58.341722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-085f1c76-8ca9-43ff-8868-88d68f406e83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:58.408181 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f1c76-8ca9-43ff-8868-88d68f406e83 service cinder] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:58.412676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74632e1-fb27-4d62-b821-82f22fee7617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:58.413536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74632e1-fb27-4d62-b821-82f22fee7617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:58.413815 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74632e1-fb27-4d62-b821-82f22fee7617 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:58.414110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74632e1-fb27-4d62-b821-82f22fee7617 admin admin] PATH_INFO: `/v3/users/fda1d04cf14641a2a6ed2152430ae845` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:58.416091 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74632e1-fb27-4d62-b821-82f22fee7617 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fda1d04cf14641a2a6ed2152430ae845)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:58.418002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74632e1-fb27-4d62-b821-82f22fee7617 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:58.464189 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b74632e1-fb27-4d62-b821-82f22fee7617 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fda1d04cf14641a2a6ed2152430ae845'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:58.476870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b74632e1-fb27-4d62-b821-82f22fee7617 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fda1d04cf14641a2a6ed2152430ae845'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:58.486358 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b74632e1-fb27-4d62-b821-82f22fee7617 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fda1d04cf14641a2a6ed2152430ae845'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:58.487073 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b74632e1-fb27-4d62-b821-82f22fee7617 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fda1d04cf14641a2a6ed2152430ae845'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:58.487674 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b74632e1-fb27-4d62-b821-82f22fee7617 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fda1d04cf14641a2a6ed2152430ae845'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:58.500379 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1022/2064] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:57:58 2022] DELETE /identity/v3/users/fda1d04cf14641a2a6ed2152430ae845 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:58.505016 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305d5b2-29f1-493c-bfd5-02ea64f5caf1 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:58.505528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305d5b2-29f1-493c-bfd5-02ea64f5caf1 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:58.505622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305d5b2-29f1-493c-bfd5-02ea64f5caf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:58.523201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-d305d5b2-29f1-493c-bfd5-02ea64f5caf1 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:58.524404 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d305d5b2-29f1-493c-bfd5-02ea64f5caf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:58.585108 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1043/2065] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:58.595806 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74632e1-fb27-4d62-b821-82f22fee7617 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:58.597240 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-b74632e1-fb27-4d62-b821-82f22fee7617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:58.658806 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74632e1-fb27-4d62-b821-82f22fee7617 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:58.662542 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f11779-14bf-4168-aa26-77277b9125a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:58.663197 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f11779-14bf-4168-aa26-77277b9125a0 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:58.663479 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f11779-14bf-4168-aa26-77277b9125a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:58.663771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f11779-14bf-4168-aa26-77277b9125a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:58.674253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f11779-14bf-4168-aa26-77277b9125a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:58.675743 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f11779-14bf-4168-aa26-77277b9125a0 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:58.681942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:58.687828 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1023/2066] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:57:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:57:58.973938 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305d5b2-29f1-493c-bfd5-02ea64f5caf1 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:58.975573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d305d5b2-29f1-493c-bfd5-02ea64f5caf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:58.994110 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305d5b2-29f1-493c-bfd5-02ea64f5caf1 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:58.999534 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1e3c8-4c54-4725-9683-ade1126144c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:59.000420 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1e3c8-4c54-4725-9683-ade1126144c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:59.000704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1e3c8-4c54-4725-9683-ade1126144c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:59.000990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1e3c8-4c54-4725-9683-ade1126144c8 admin admin] PATH_INFO: `/v3/projects/16edf1463a93425298357fc12752c5a9` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:59.002387 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1e3c8-4c54-4725-9683-ade1126144c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16edf1463a93425298357fc12752c5a9)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:59.005768 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1e3c8-4c54-4725-9683-ade1126144c8 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:59.055752 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d0c1e3c8-4c54-4725-9683-ade1126144c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16edf1463a93425298357fc12752c5a9', 'request_id': 'req-d0c1e3c8-4c54-4725-9683-ade1126144c8'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:59.061980 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d0c1e3c8-4c54-4725-9683-ade1126144c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16edf1463a93425298357fc12752c5a9', 'request_id': 'req-d0c1e3c8-4c54-4725-9683-ade1126144c8'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:59.068526 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d0c1e3c8-4c54-4725-9683-ade1126144c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16edf1463a93425298357fc12752c5a9', 'request_id': 'req-d0c1e3c8-4c54-4725-9683-ade1126144c8'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:59.084499 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d0c1e3c8-4c54-4725-9683-ade1126144c8 admin admin] The token cache is being invalidate because project 16edf1463a93425298357fc12752c5a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:57:59.084793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d0c1e3c8-4c54-4725-9683-ade1126144c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:59.085409 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d0c1e3c8-4c54-4725-9683-ade1126144c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:57:59.087496 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1044/2067] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:57:58 2022] DELETE /identity/v3/projects/16edf1463a93425298357fc12752c5a9 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:59.093385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577a460-5a8d-4ace-aa57-123c4e92bfa0 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:59.093627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577a460-5a8d-4ace-aa57-123c4e92bfa0 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:59.093993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577a460-5a8d-4ace-aa57-123c4e92bfa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:59.120473 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-a577a460-5a8d-4ace-aa57-123c4e92bfa0 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:59.121694 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-a577a460-5a8d-4ace-aa57-123c4e92bfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:59.185188 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1024/2068] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:59.192733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb34bb-7c4f-4caf-a900-be7cc259ea84 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:59.193030 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb34bb-7c4f-4caf-a900-be7cc259ea84 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:59.193317 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb34bb-7c4f-4caf-a900-be7cc259ea84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:59.217339 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-a1eb34bb-7c4f-4caf-a900-be7cc259ea84 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:59.218610 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a1eb34bb-7c4f-4caf-a900-be7cc259ea84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:59.227091 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1045/2069] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:59.231016 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a577a460-5a8d-4ace-aa57-123c4e92bfa0 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:59.241481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a577a460-5a8d-4ace-aa57-123c4e92bfa0 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:59.246998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c1a81-1a4e-454f-a9c3-ed223eb444eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:59.247803 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c1a81-1a4e-454f-a9c3-ed223eb444eb admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:59.248088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c1a81-1a4e-454f-a9c3-ed223eb444eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:59.248363 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c1a81-1a4e-454f-a9c3-ed223eb444eb admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:59.249101 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c1a81-1a4e-454f-a9c3-ed223eb444eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:57:59.249482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c1a81-1a4e-454f-a9c3-ed223eb444eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:59.251163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c1a81-1a4e-454f-a9c3-ed223eb444eb admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:59.265081 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1025/2070] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:57:59.269748 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb34bb-7c4f-4caf-a900-be7cc259ea84 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:59.283291 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb34bb-7c4f-4caf-a900-be7cc259ea84 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:59.287716 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190c9c4-aadf-4630-88d6-d750138b2bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:59.288393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190c9c4-aadf-4630-88d6-d750138b2bf5 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:59.288679 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190c9c4-aadf-4630-88d6-d750138b2bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:59.289017 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190c9c4-aadf-4630-88d6-d750138b2bf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:59.289665 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7190c9c4-aadf-4630-88d6-d750138b2bf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:59.291741 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7190c9c4-aadf-4630-88d6-d750138b2bf5 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:59.325472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1046/2071] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:57:59 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:57:59.326918 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c1a81-1a4e-454f-a9c3-ed223eb444eb admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:59.340417 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c1a81-1a4e-454f-a9c3-ed223eb444eb admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:59.403707 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb98eca-9a81-4533-abee-5073a101bc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:59.404626 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb98eca-9a81-4533-abee-5073a101bc28 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:59.404923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb98eca-9a81-4533-abee-5073a101bc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:59.405207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb98eca-9a81-4533-abee-5073a101bc28 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:59.405897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb98eca-9a81-4533-abee-5073a101bc28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:59.407896 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb98eca-9a81-4533-abee-5073a101bc28 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:59.427696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1026/2072] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:57:59 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:57:59.431924 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190c9c4-aadf-4630-88d6-d750138b2bf5 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:59.443231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190c9c4-aadf-4630-88d6-d750138b2bf5 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:59.448755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e8afd-a455-4085-951c-4a174df3edb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:59.449446 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e8afd-a455-4085-951c-4a174df3edb3 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:59.449730 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e8afd-a455-4085-951c-4a174df3edb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:59.450033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e8afd-a455-4085-951c-4a174df3edb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:59.450774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e8afd-a455-4085-951c-4a174df3edb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:59.452396 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e8afd-a455-4085-951c-4a174df3edb3 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:59.464192 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1047/2073] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:57:59.468518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb98eca-9a81-4533-abee-5073a101bc28 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:59.478692 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb98eca-9a81-4533-abee-5073a101bc28 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:59.483361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd0909-1a18-4bbc-a032-8a3a226c989a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:59.484010 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd0909-1a18-4bbc-a032-8a3a226c989a admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:59.484288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd0909-1a18-4bbc-a032-8a3a226c989a admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:59.484558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd0909-1a18-4bbc-a032-8a3a226c989a admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:59.485280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd0909-1a18-4bbc-a032-8a3a226c989a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:59.486993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd0909-1a18-4bbc-a032-8a3a226c989a admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:59.498386 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1027/2074] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:57:59.502972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e8afd-a455-4085-951c-4a174df3edb3 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:59.514272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e8afd-a455-4085-951c-4a174df3edb3 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:59.519744 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a01ae4-14bf-46bd-9612-24fabe8fd1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:59.520625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a01ae4-14bf-46bd-9612-24fabe8fd1cb admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:59.520910 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a01ae4-14bf-46bd-9612-24fabe8fd1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:59.521186 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a01ae4-14bf-46bd-9612-24fabe8fd1cb admin admin] PATH_INFO: `/v3/projects/dfbdd7703e9c40b9ab0f5c5b9712ad6a/users/da5e45eaa8c5409592fd9bb2a3fb45e5/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:59.532785 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:59.537961 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a01ae4-14bf-46bd-9612-24fabe8fd1cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfbdd7703e9c40b9ab0f5c5b9712ad6a, user_id=da5e45eaa8c5409592fd9bb2a3fb45e5, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:59.541206 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a01ae4-14bf-46bd-9612-24fabe8fd1cb admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:59.561991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1048/2075] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:57:59 2022] PUT /identity/v3/projects/dfbdd7703e9c40b9ab0f5c5b9712ad6a/users/da5e45eaa8c5409592fd9bb2a3fb45e5/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:57:59.566780 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6715f-d83a-4ffb-8dd7-3b280624e730 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:59.567071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6715f-d83a-4ffb-8dd7-3b280624e730 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:59.567354 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6715f-d83a-4ffb-8dd7-3b280624e730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:59.595175 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-6fd6715f-d83a-4ffb-8dd7-3b280624e730 None None] MFA Rules not processed for user `da5e45eaa8c5409592fd9bb2a3fb45e5`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:57:59.596351 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-6fd6715f-d83a-4ffb-8dd7-3b280624e730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:59.654520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:59.665231 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1028/2076] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:57:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:57:59.677168 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a01ae4-14bf-46bd-9612-24fabe8fd1cb admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:59.678916 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-69a01ae4-14bf-46bd-9612-24fabe8fd1cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:59.753980 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a01ae4-14bf-46bd-9612-24fabe8fd1cb admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:59.760375 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997bdcad-a5ac-408f-b334-0887c5f77736 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:59.761023 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997bdcad-a5ac-408f-b334-0887c5f77736 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:59.761310 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997bdcad-a5ac-408f-b334-0887c5f77736 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:59.761593 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997bdcad-a5ac-408f-b334-0887c5f77736 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:59.769796 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997bdcad-a5ac-408f-b334-0887c5f77736 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:59.771470 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997bdcad-a5ac-408f-b334-0887c5f77736 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:59.780418 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1049/2077] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:57:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:57:59.986837 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6715f-d83a-4ffb-8dd7-3b280624e730 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:00.052618 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6715f-d83a-4ffb-8dd7-3b280624e730 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:00.057869 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf2b6a-8a84-4457-94bc-44fe595b25d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:00.058810 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf2b6a-8a84-4457-94bc-44fe595b25d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:00.059130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf2b6a-8a84-4457-94bc-44fe595b25d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:00.059404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf2b6a-8a84-4457-94bc-44fe595b25d6 admin admin] PATH_INFO: `/v3/users/da5e45eaa8c5409592fd9bb2a3fb45e5` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:00.060730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbf2b6a-8a84-4457-94bc-44fe595b25d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da5e45eaa8c5409592fd9bb2a3fb45e5)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:00.062810 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbf2b6a-8a84-4457-94bc-44fe595b25d6 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:00.113145 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-5dbf2b6a-8a84-4457-94bc-44fe595b25d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da5e45eaa8c5409592fd9bb2a3fb45e5'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:00.127843 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-5dbf2b6a-8a84-4457-94bc-44fe595b25d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da5e45eaa8c5409592fd9bb2a3fb45e5'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:00.135546 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-5dbf2b6a-8a84-4457-94bc-44fe595b25d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da5e45eaa8c5409592fd9bb2a3fb45e5'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:00.136182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-5dbf2b6a-8a84-4457-94bc-44fe595b25d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da5e45eaa8c5409592fd9bb2a3fb45e5'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:00.136847 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-5dbf2b6a-8a84-4457-94bc-44fe595b25d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da5e45eaa8c5409592fd9bb2a3fb45e5'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:00.149715 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1029/2078] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:57:59 2022] DELETE /identity/v3/users/da5e45eaa8c5409592fd9bb2a3fb45e5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:00.154609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66389165-a038-4c3c-a24b-688f0ea4b590 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:00.155069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66389165-a038-4c3c-a24b-688f0ea4b590 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:00.155363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66389165-a038-4c3c-a24b-688f0ea4b590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:00.177313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-66389165-a038-4c3c-a24b-688f0ea4b590 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:00.177313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-66389165-a038-4c3c-a24b-688f0ea4b590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:00.238694 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1050/2079] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:00.251614 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf2b6a-8a84-4457-94bc-44fe595b25d6 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:00.254463 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-5dbf2b6a-8a84-4457-94bc-44fe595b25d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:00.315251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf2b6a-8a84-4457-94bc-44fe595b25d6 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:00.319372 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f8dce-4933-463f-b172-fe786ec5bee2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:00.320126 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f8dce-4933-463f-b172-fe786ec5bee2 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:00.320400 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f8dce-4933-463f-b172-fe786ec5bee2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:00.320666 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f8dce-4933-463f-b172-fe786ec5bee2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:00.330548 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f8dce-4933-463f-b172-fe786ec5bee2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:00.331966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f8dce-4933-463f-b172-fe786ec5bee2 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:00.342337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1030/2080] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:00.655067 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66389165-a038-4c3c-a24b-688f0ea4b590 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:00.656534 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-66389165-a038-4c3c-a24b-688f0ea4b590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:00.668677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66389165-a038-4c3c-a24b-688f0ea4b590 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:00.677621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21182f-d1ff-46b0-a1a8-a3dd441efa5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:00.678524 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21182f-d1ff-46b0-a1a8-a3dd441efa5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:00.678813 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21182f-d1ff-46b0-a1a8-a3dd441efa5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:00.679093 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21182f-d1ff-46b0-a1a8-a3dd441efa5f admin admin] PATH_INFO: `/v3/projects/dfbdd7703e9c40b9ab0f5c5b9712ad6a` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:00.680319 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c21182f-d1ff-46b0-a1a8-a3dd441efa5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfbdd7703e9c40b9ab0f5c5b9712ad6a)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:00.682349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c21182f-d1ff-46b0-a1a8-a3dd441efa5f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:00.735571 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3c21182f-d1ff-46b0-a1a8-a3dd441efa5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfbdd7703e9c40b9ab0f5c5b9712ad6a', 'request_id': 'req-3c21182f-d1ff-46b0-a1a8-a3dd441efa5f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:00.740944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3c21182f-d1ff-46b0-a1a8-a3dd441efa5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfbdd7703e9c40b9ab0f5c5b9712ad6a', 'request_id': 'req-3c21182f-d1ff-46b0-a1a8-a3dd441efa5f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:00.747230 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3c21182f-d1ff-46b0-a1a8-a3dd441efa5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfbdd7703e9c40b9ab0f5c5b9712ad6a', 'request_id': 'req-3c21182f-d1ff-46b0-a1a8-a3dd441efa5f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:00.755752 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:00.761629 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3c21182f-d1ff-46b0-a1a8-a3dd441efa5f admin admin] The token cache is being invalidate because project dfbdd7703e9c40b9ab0f5c5b9712ad6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:00.762181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3c21182f-d1ff-46b0-a1a8-a3dd441efa5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:00.762917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3c21182f-d1ff-46b0-a1a8-a3dd441efa5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:00.765280 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1051/2081] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:58:00 2022] DELETE /identity/v3/projects/dfbdd7703e9c40b9ab0f5c5b9712ad6a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:00.770585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d36ecc-80eb-4af2-9692-b5c2651e6989 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:00.770863 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d36ecc-80eb-4af2-9692-b5c2651e6989 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:00.771132 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d36ecc-80eb-4af2-9692-b5c2651e6989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:00.789450 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-54d36ecc-80eb-4af2-9692-b5c2651e6989 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:00.790737 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-54d36ecc-80eb-4af2-9692-b5c2651e6989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:00.803494 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:00.854956 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1031/2082] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:00.864374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd9389-cbe7-4dda-a7fc-a2149bbb5326 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:00.864665 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd9389-cbe7-4dda-a7fc-a2149bbb5326 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:00.864951 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd9389-cbe7-4dda-a7fc-a2149bbb5326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:00.884824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-27fd9389-cbe7-4dda-a7fc-a2149bbb5326 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:00.886060 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-27fd9389-cbe7-4dda-a7fc-a2149bbb5326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:00.894246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1052/2083] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:00.898782 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d36ecc-80eb-4af2-9692-b5c2651e6989 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:00.909194 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d36ecc-80eb-4af2-9692-b5c2651e6989 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:00.913825 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1e51e-a53e-4bc3-b804-0f368202383c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:00.914504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1e51e-a53e-4bc3-b804-0f368202383c admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:00.914784 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1e51e-a53e-4bc3-b804-0f368202383c admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:00.915057 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1e51e-a53e-4bc3-b804-0f368202383c admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:00.915778 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1e51e-a53e-4bc3-b804-0f368202383c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:58:00.916074 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1e51e-a53e-4bc3-b804-0f368202383c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:00.917659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1e51e-a53e-4bc3-b804-0f368202383c admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:00.929334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1032/2084] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:58:00.933844 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd9389-cbe7-4dda-a7fc-a2149bbb5326 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:00.944231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd9389-cbe7-4dda-a7fc-a2149bbb5326 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:00.949442 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cc656-2c03-42b0-b0b9-093cb617ab6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:00.950151 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cc656-2c03-42b0-b0b9-093cb617ab6b admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:00.950436 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cc656-2c03-42b0-b0b9-093cb617ab6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:00.950714 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cc656-2c03-42b0-b0b9-093cb617ab6b admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:00.951420 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7cc656-2c03-42b0-b0b9-093cb617ab6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:00.953397 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7cc656-2c03-42b0-b0b9-093cb617ab6b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:00.980528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1053/2085] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:58:00 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:00.984753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1e51e-a53e-4bc3-b804-0f368202383c admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:00.995627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1e51e-a53e-4bc3-b804-0f368202383c admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:01.059808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32545463-533a-4641-9595-aa6172f27ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:01.060487 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32545463-533a-4641-9595-aa6172f27ba3 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:01.060772 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32545463-533a-4641-9595-aa6172f27ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:01.061048 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32545463-533a-4641-9595-aa6172f27ba3 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:01.061735 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32545463-533a-4641-9595-aa6172f27ba3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:01.063703 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32545463-533a-4641-9595-aa6172f27ba3 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:01.083986 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1033/2086] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:58:00 2022] POST /identity/v3/users => generated 282 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:01.088113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cc656-2c03-42b0-b0b9-093cb617ab6b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:01.098834 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cc656-2c03-42b0-b0b9-093cb617ab6b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:01.103795 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc661fff-9fa5-484c-bece-0d8be13d24f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:01.104459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc661fff-9fa5-484c-bece-0d8be13d24f1 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:01.104748 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc661fff-9fa5-484c-bece-0d8be13d24f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:01.105028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc661fff-9fa5-484c-bece-0d8be13d24f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:01.105784 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc661fff-9fa5-484c-bece-0d8be13d24f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:01.107578 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc661fff-9fa5-484c-bece-0d8be13d24f1 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:01.119803 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1054/2087] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:01.124106 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32545463-533a-4641-9595-aa6172f27ba3 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:01.134906 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32545463-533a-4641-9595-aa6172f27ba3 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:01.139611 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1999e2a-61e0-4b94-8a97-0db0109515c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:01.140277 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1999e2a-61e0-4b94-8a97-0db0109515c8 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:01.140563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1999e2a-61e0-4b94-8a97-0db0109515c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:01.140844 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1999e2a-61e0-4b94-8a97-0db0109515c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:01.141871 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1999e2a-61e0-4b94-8a97-0db0109515c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:01.143584 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1999e2a-61e0-4b94-8a97-0db0109515c8 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:01.154456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1034/2088] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:01.159236 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc661fff-9fa5-484c-bece-0d8be13d24f1 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:01.169400 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc661fff-9fa5-484c-bece-0d8be13d24f1 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:01.174055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417428b-3ce9-4794-a1be-1fc96739ea2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:01.174935 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417428b-3ce9-4794-a1be-1fc96739ea2c admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:01.175216 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417428b-3ce9-4794-a1be-1fc96739ea2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:01.175490 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417428b-3ce9-4794-a1be-1fc96739ea2c admin admin] PATH_INFO: `/v3/projects/18eced2fb7c7429ca0897582858df1f2/users/5e5b359030154a6a8c4058831e9c55f6/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:01.190247 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417428b-3ce9-4794-a1be-1fc96739ea2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=18eced2fb7c7429ca0897582858df1f2, user_id=5e5b359030154a6a8c4058831e9c55f6, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:01.192937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417428b-3ce9-4794-a1be-1fc96739ea2c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:01.207266 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1055/2089] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:58:01 2022] PUT /identity/v3/projects/18eced2fb7c7429ca0897582858df1f2/users/5e5b359030154a6a8c4058831e9c55f6/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:01.217820 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0c74f-5928-423e-8a0d-d8fd930cd6bc None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:01.218157 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0c74f-5928-423e-8a0d-d8fd930cd6bc None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:01.218437 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0c74f-5928-423e-8a0d-d8fd930cd6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:01.247231 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-f4d0c74f-5928-423e-8a0d-d8fd930cd6bc None None] MFA Rules not processed for user `5e5b359030154a6a8c4058831e9c55f6`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:01.248458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f4d0c74f-5928-423e-8a0d-d8fd930cd6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:01.315017 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1035/2090] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:01 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:01.319656 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417428b-3ce9-4794-a1be-1fc96739ea2c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:01.385934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417428b-3ce9-4794-a1be-1fc96739ea2c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:01.390990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4146887-c9b7-47ea-917f-77a837b04e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:01.391946 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4146887-c9b7-47ea-917f-77a837b04e05 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:01.392226 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4146887-c9b7-47ea-917f-77a837b04e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:01.392499 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4146887-c9b7-47ea-917f-77a837b04e05 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:01.393212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4146887-c9b7-47ea-917f-77a837b04e05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:01.395328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4146887-c9b7-47ea-917f-77a837b04e05 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:01.421383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1056/2091] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:58:01 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:01.425288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d0c74f-5928-423e-8a0d-d8fd930cd6bc None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:01.436772 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d0c74f-5928-423e-8a0d-d8fd930cd6bc None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:01.447494 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:01.498983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef49d32-9d19-4842-9238-43e7ebefeb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:01.500070 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef49d32-9d19-4842-9238-43e7ebefeb3e admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:01.500348 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef49d32-9d19-4842-9238-43e7ebefeb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:01.500619 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef49d32-9d19-4842-9238-43e7ebefeb3e admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:01.501317 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef49d32-9d19-4842-9238-43e7ebefeb3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:01.503270 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef49d32-9d19-4842-9238-43e7ebefeb3e admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:01.524219 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1036/2092] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:58:01 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:01.528849 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4146887-c9b7-47ea-917f-77a837b04e05 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:01.539369 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4146887-c9b7-47ea-917f-77a837b04e05 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:01.544080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b047579-393b-43fe-ba6b-95d4e9747174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:01.544753 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b047579-393b-43fe-ba6b-95d4e9747174 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:01.545111 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b047579-393b-43fe-ba6b-95d4e9747174 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:01.545423 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b047579-393b-43fe-ba6b-95d4e9747174 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:01.546199 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b047579-393b-43fe-ba6b-95d4e9747174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:01.547824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b047579-393b-43fe-ba6b-95d4e9747174 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:01.559881 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1057/2093] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:58:01.564648 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef49d32-9d19-4842-9238-43e7ebefeb3e admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:01.576130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef49d32-9d19-4842-9238-43e7ebefeb3e admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:01.580857 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8234ea2-58f6-443c-a049-d4bf5341e7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:01.581736 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8234ea2-58f6-443c-a049-d4bf5341e7e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:01.582038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8234ea2-58f6-443c-a049-d4bf5341e7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:01.582319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8234ea2-58f6-443c-a049-d4bf5341e7e6 admin admin] PATH_INFO: `/v3/projects/ab687c2b2bd94cc3ab6dd97b1795832a/users/6c44c08b85d14ef79f553d0619a76c6d/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:01.598575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8234ea2-58f6-443c-a049-d4bf5341e7e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab687c2b2bd94cc3ab6dd97b1795832a, user_id=6c44c08b85d14ef79f553d0619a76c6d, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:01.601282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8234ea2-58f6-443c-a049-d4bf5341e7e6 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:01.616420 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1037/2094] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:58:01 2022] PUT /identity/v3/projects/ab687c2b2bd94cc3ab6dd97b1795832a/users/6c44c08b85d14ef79f553d0619a76c6d/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:58:01.620760 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b047579-393b-43fe-ba6b-95d4e9747174 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:01.634969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b047579-393b-43fe-ba6b-95d4e9747174 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:01.701366 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f25c8-64f4-475e-a42a-48a02797a5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:01.702528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f25c8-64f4-475e-a42a-48a02797a5d8 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:01.702875 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f25c8-64f4-475e-a42a-48a02797a5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:01.703160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f25c8-64f4-475e-a42a-48a02797a5d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:01.703899 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f25c8-64f4-475e-a42a-48a02797a5d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:01.705744 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f25c8-64f4-475e-a42a-48a02797a5d8 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:01.717076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1058/2095] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:01.721258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8234ea2-58f6-443c-a049-d4bf5341e7e6 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:01.738931 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8234ea2-58f6-443c-a049-d4bf5341e7e6 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:01.744092 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec77ee-1f13-408e-848f-95dae816fc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:01.744971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec77ee-1f13-408e-848f-95dae816fc10 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:01.745250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec77ee-1f13-408e-848f-95dae816fc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:01.745548 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec77ee-1f13-408e-848f-95dae816fc10 admin admin] PATH_INFO: `/v3/projects/ab687c2b2bd94cc3ab6dd97b1795832a/users/6c44c08b85d14ef79f553d0619a76c6d/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:01.747552 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec77ee-1f13-408e-848f-95dae816fc10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab687c2b2bd94cc3ab6dd97b1795832a, user_id=6c44c08b85d14ef79f553d0619a76c6d, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:01.750248 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec77ee-1f13-408e-848f-95dae816fc10 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:01.761129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1038/2096] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:58:01 2022] PUT /identity/v3/projects/ab687c2b2bd94cc3ab6dd97b1795832a/users/6c44c08b85d14ef79f553d0619a76c6d/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:58:01.771751 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915b053-fd1a-4bfb-87ff-2bf658b4a564 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:01.772046 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915b053-fd1a-4bfb-87ff-2bf658b4a564 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:01.772357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915b053-fd1a-4bfb-87ff-2bf658b4a564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:01.779124 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:01.800021 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-1915b053-fd1a-4bfb-87ff-2bf658b4a564 None None] MFA Rules not processed for user `6c44c08b85d14ef79f553d0619a76c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:01.801238 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1915b053-fd1a-4bfb-87ff-2bf658b4a564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:01.876122 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1059/2097] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:01 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:01.888361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec77ee-1f13-408e-848f-95dae816fc10 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:01.890094 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-adec77ee-1f13-408e-848f-95dae816fc10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:01.952260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec77ee-1f13-408e-848f-95dae816fc10 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:01.955846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb8ff9-953d-4fda-a222-3120ced3662b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:01.956487 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb8ff9-953d-4fda-a222-3120ced3662b service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:01.956762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb8ff9-953d-4fda-a222-3120ced3662b service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:01.957031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb8ff9-953d-4fda-a222-3120ced3662b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:01.966217 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eb8ff9-953d-4fda-a222-3120ced3662b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:01.967656 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eb8ff9-953d-4fda-a222-3120ced3662b service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:01.978289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1039/2098] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:01 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:02.402154 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915b053-fd1a-4bfb-87ff-2bf658b4a564 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:02.479485 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915b053-fd1a-4bfb-87ff-2bf658b4a564 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:02.493582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79b99a-7a6f-40be-b7fb-3021ca1bd485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:02.494705 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79b99a-7a6f-40be-b7fb-3021ca1bd485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:02.495130 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79b99a-7a6f-40be-b7fb-3021ca1bd485 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:02.495490 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79b99a-7a6f-40be-b7fb-3021ca1bd485 admin admin] PATH_INFO: `/v3/users/5e5b359030154a6a8c4058831e9c55f6` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:02.496720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79b99a-7a6f-40be-b7fb-3021ca1bd485 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e5b359030154a6a8c4058831e9c55f6)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:02.500699 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79b99a-7a6f-40be-b7fb-3021ca1bd485 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:02.570178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2c79b99a-7a6f-40be-b7fb-3021ca1bd485 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e5b359030154a6a8c4058831e9c55f6'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:02.588407 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2c79b99a-7a6f-40be-b7fb-3021ca1bd485 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e5b359030154a6a8c4058831e9c55f6'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:02.599219 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2c79b99a-7a6f-40be-b7fb-3021ca1bd485 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e5b359030154a6a8c4058831e9c55f6'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:02.599851 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2c79b99a-7a6f-40be-b7fb-3021ca1bd485 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e5b359030154a6a8c4058831e9c55f6'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:02.600676 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2c79b99a-7a6f-40be-b7fb-3021ca1bd485 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e5b359030154a6a8c4058831e9c55f6'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:02.627782 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1060/2099] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:58:02 2022] DELETE /identity/v3/users/5e5b359030154a6a8c4058831e9c55f6 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:02.632680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad9bce-927a-4357-9c90-303ac828f87a None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:02.633189 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad9bce-927a-4357-9c90-303ac828f87a None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:02.633415 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad9bce-927a-4357-9c90-303ac828f87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:02.656280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-1fad9bce-927a-4357-9c90-303ac828f87a None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:02.657495 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-1fad9bce-927a-4357-9c90-303ac828f87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:02.720585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1040/2100] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:02.733701 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79b99a-7a6f-40be-b7fb-3021ca1bd485 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:02.734555 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2c79b99a-7a6f-40be-b7fb-3021ca1bd485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:02.748987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:02.815194 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79b99a-7a6f-40be-b7fb-3021ca1bd485 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:02.819073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b34b9-4ab6-465d-868c-bd1b23807488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:02.819682 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b34b9-4ab6-465d-868c-bd1b23807488 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:02.820023 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b34b9-4ab6-465d-868c-bd1b23807488 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:02.820368 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b34b9-4ab6-465d-868c-bd1b23807488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:02.831249 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b34b9-4ab6-465d-868c-bd1b23807488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:02.832733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b34b9-4ab6-465d-868c-bd1b23807488 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:02.843877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1061/2101] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:03.150460 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad9bce-927a-4357-9c90-303ac828f87a None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:03.152082 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-1fad9bce-927a-4357-9c90-303ac828f87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:03.166565 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad9bce-927a-4357-9c90-303ac828f87a None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:03.171441 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9501e-5df1-4a9a-8870-14160682668d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:03.172479 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9501e-5df1-4a9a-8870-14160682668d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:03.172898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9501e-5df1-4a9a-8870-14160682668d admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:03.173209 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9501e-5df1-4a9a-8870-14160682668d admin admin] PATH_INFO: `/v3/projects/18eced2fb7c7429ca0897582858df1f2` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:03.175205 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9501e-5df1-4a9a-8870-14160682668d admin admin] RBAC: Authorizing `identity:delete_project(project_id=18eced2fb7c7429ca0897582858df1f2)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:03.177085 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9501e-5df1-4a9a-8870-14160682668d admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:03.179743 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:03.227925 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-1bb9501e-5df1-4a9a-8870-14160682668d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18eced2fb7c7429ca0897582858df1f2', 'request_id': 'req-1bb9501e-5df1-4a9a-8870-14160682668d'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:03.234033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-1bb9501e-5df1-4a9a-8870-14160682668d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18eced2fb7c7429ca0897582858df1f2', 'request_id': 'req-1bb9501e-5df1-4a9a-8870-14160682668d'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:03.240368 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-1bb9501e-5df1-4a9a-8870-14160682668d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18eced2fb7c7429ca0897582858df1f2', 'request_id': 'req-1bb9501e-5df1-4a9a-8870-14160682668d'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:03.254453 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-1bb9501e-5df1-4a9a-8870-14160682668d admin admin] The token cache is being invalidate because project 18eced2fb7c7429ca0897582858df1f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:03.254740 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-1bb9501e-5df1-4a9a-8870-14160682668d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:03.255383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-1bb9501e-5df1-4a9a-8870-14160682668d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:03.256964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1041/2102] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:58:03 2022] DELETE /identity/v3/projects/18eced2fb7c7429ca0897582858df1f2 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:03.261953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b34b9-4ab6-465d-868c-bd1b23807488 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:03.272154 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b34b9-4ab6-465d-868c-bd1b23807488 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:03.330881 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2138dc3-4657-4bc6-9ced-7d4be79267d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:03.331766 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2138dc3-4657-4bc6-9ced-7d4be79267d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:03.332089 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2138dc3-4657-4bc6-9ced-7d4be79267d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:03.332424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2138dc3-4657-4bc6-9ced-7d4be79267d2 admin admin] PATH_INFO: `/v3/users/6c44c08b85d14ef79f553d0619a76c6d` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:03.333696 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2138dc3-4657-4bc6-9ced-7d4be79267d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c44c08b85d14ef79f553d0619a76c6d)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:03.335801 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2138dc3-4657-4bc6-9ced-7d4be79267d2 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:03.381616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c2138dc3-4657-4bc6-9ced-7d4be79267d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c44c08b85d14ef79f553d0619a76c6d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:03.396109 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c2138dc3-4657-4bc6-9ced-7d4be79267d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c44c08b85d14ef79f553d0619a76c6d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:03.403108 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c2138dc3-4657-4bc6-9ced-7d4be79267d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c44c08b85d14ef79f553d0619a76c6d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:03.403978 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c2138dc3-4657-4bc6-9ced-7d4be79267d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c44c08b85d14ef79f553d0619a76c6d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:03.404597 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c2138dc3-4657-4bc6-9ced-7d4be79267d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c44c08b85d14ef79f553d0619a76c6d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:03.417631 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1062/2103] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:58:03 2022] DELETE /identity/v3/users/6c44c08b85d14ef79f553d0619a76c6d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:03.702543 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9501e-5df1-4a9a-8870-14160682668d admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:03.705082 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-1bb9501e-5df1-4a9a-8870-14160682668d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:03.720868 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9501e-5df1-4a9a-8870-14160682668d admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:03.782334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5235d-56c0-4ad5-8efc-b5efbe66484d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:03.783114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5235d-56c0-4ad5-8efc-b5efbe66484d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:03.783395 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5235d-56c0-4ad5-8efc-b5efbe66484d admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:03.783666 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5235d-56c0-4ad5-8efc-b5efbe66484d admin admin] PATH_INFO: `/v3/projects/ab687c2b2bd94cc3ab6dd97b1795832a` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:03.784876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5235d-56c0-4ad5-8efc-b5efbe66484d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab687c2b2bd94cc3ab6dd97b1795832a)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:03.788144 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5235d-56c0-4ad5-8efc-b5efbe66484d admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:03.841462 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-e2e5235d-56c0-4ad5-8efc-b5efbe66484d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab687c2b2bd94cc3ab6dd97b1795832a', 'request_id': 'req-e2e5235d-56c0-4ad5-8efc-b5efbe66484d'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:03.848435 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-e2e5235d-56c0-4ad5-8efc-b5efbe66484d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab687c2b2bd94cc3ab6dd97b1795832a', 'request_id': 'req-e2e5235d-56c0-4ad5-8efc-b5efbe66484d'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:03.855510 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-e2e5235d-56c0-4ad5-8efc-b5efbe66484d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab687c2b2bd94cc3ab6dd97b1795832a', 'request_id': 'req-e2e5235d-56c0-4ad5-8efc-b5efbe66484d'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:03.869409 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-e2e5235d-56c0-4ad5-8efc-b5efbe66484d admin admin] The token cache is being invalidate because project ab687c2b2bd94cc3ab6dd97b1795832a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:03.869713 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-e2e5235d-56c0-4ad5-8efc-b5efbe66484d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:03.870387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-e2e5235d-56c0-4ad5-8efc-b5efbe66484d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:03.872141 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1042/2104] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:58:03 2022] DELETE /identity/v3/projects/ab687c2b2bd94cc3ab6dd97b1795832a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:03.880097 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712c85c-de99-4992-acee-72c780eff326 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:03.880736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712c85c-de99-4992-acee-72c780eff326 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:03.881312 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712c85c-de99-4992-acee-72c780eff326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:03.903172 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-a712c85c-de99-4992-acee-72c780eff326 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:03.904387 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a712c85c-de99-4992-acee-72c780eff326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:03.935287 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:03.976819 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1063/2105] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:03.986526 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a3faa-8c80-4bcc-ae2b-bb787bc1a7e4 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:03.986806 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a3faa-8c80-4bcc-ae2b-bb787bc1a7e4 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:03.987076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a3faa-8c80-4bcc-ae2b-bb787bc1a7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:04.006968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-b74a3faa-8c80-4bcc-ae2b-bb787bc1a7e4 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:04.008483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-b74a3faa-8c80-4bcc-ae2b-bb787bc1a7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:04.021640 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1043/2106] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:04.026720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712c85c-de99-4992-acee-72c780eff326 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:04.036577 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712c85c-de99-4992-acee-72c780eff326 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:04.041794 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ea9d6-2e64-453a-b8e3-4363be0e50af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:04.042491 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ea9d6-2e64-453a-b8e3-4363be0e50af admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:04.042782 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ea9d6-2e64-453a-b8e3-4363be0e50af admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:04.043204 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ea9d6-2e64-453a-b8e3-4363be0e50af admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:04.043970 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ea9d6-2e64-453a-b8e3-4363be0e50af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:58:04.044283 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ea9d6-2e64-453a-b8e3-4363be0e50af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:04.045898 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ea9d6-2e64-453a-b8e3-4363be0e50af admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:04.060647 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1064/2107] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:58:04.065107 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a3faa-8c80-4bcc-ae2b-bb787bc1a7e4 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:04.078216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a3faa-8c80-4bcc-ae2b-bb787bc1a7e4 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:04.083729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a64170-181d-4147-b193-efcfc2c45ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:04.084396 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a64170-181d-4147-b193-efcfc2c45ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:04.084676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a64170-181d-4147-b193-efcfc2c45ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:04.084951 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a64170-181d-4147-b193-efcfc2c45ae5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:04.085642 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a64170-181d-4147-b193-efcfc2c45ae5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:04.089718 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a64170-181d-4147-b193-efcfc2c45ae5 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:04.107818 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:04.123056 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1044/2108] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:58:04 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:04.127386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ea9d6-2e64-453a-b8e3-4363be0e50af admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:04.139548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ea9d6-2e64-453a-b8e3-4363be0e50af admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:04.212348 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65edd05-9fb2-45ef-846f-abf017eccb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:04.213033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65edd05-9fb2-45ef-846f-abf017eccb39 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:04.213322 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65edd05-9fb2-45ef-846f-abf017eccb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:04.213657 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65edd05-9fb2-45ef-846f-abf017eccb39 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:04.214612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65edd05-9fb2-45ef-846f-abf017eccb39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:04.216716 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65edd05-9fb2-45ef-846f-abf017eccb39 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:04.237388 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1065/2109] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:58:04 2022] POST /identity/v3/users => generated 283 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:04.241766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a64170-181d-4147-b193-efcfc2c45ae5 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:04.252796 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a64170-181d-4147-b193-efcfc2c45ae5 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:04.258260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c22b5a-aae5-4d72-87b4-540762b5fd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:04.259018 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c22b5a-aae5-4d72-87b4-540762b5fd93 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:04.259303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c22b5a-aae5-4d72-87b4-540762b5fd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:04.259573 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c22b5a-aae5-4d72-87b4-540762b5fd93 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:04.260304 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c22b5a-aae5-4d72-87b4-540762b5fd93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:04.262033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c22b5a-aae5-4d72-87b4-540762b5fd93 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:04.275913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1045/2110] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:04.279952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65edd05-9fb2-45ef-846f-abf017eccb39 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:04.290675 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65edd05-9fb2-45ef-846f-abf017eccb39 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:04.297985 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6803e76-bb30-4b62-b92c-0e0ed4a5c0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:04.298712 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6803e76-bb30-4b62-b92c-0e0ed4a5c0d2 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:04.298998 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6803e76-bb30-4b62-b92c-0e0ed4a5c0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:04.299278 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6803e76-bb30-4b62-b92c-0e0ed4a5c0d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:04.300023 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6803e76-bb30-4b62-b92c-0e0ed4a5c0d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:04.301671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6803e76-bb30-4b62-b92c-0e0ed4a5c0d2 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:04.315906 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1066/2111] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:04.317206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c22b5a-aae5-4d72-87b4-540762b5fd93 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:04.327406 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c22b5a-aae5-4d72-87b4-540762b5fd93 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:04.332174 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408bc42-65e1-4b7c-ae04-87c3deb44f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:04.333257 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408bc42-65e1-4b7c-ae04-87c3deb44f03 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:04.334087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408bc42-65e1-4b7c-ae04-87c3deb44f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:04.334087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408bc42-65e1-4b7c-ae04-87c3deb44f03 admin admin] PATH_INFO: `/v3/projects/510f193078374bf3b129d669d4cd5866/users/214e86037f3c4d47bac76a8e0ae76e0f/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:04.350663 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b408bc42-65e1-4b7c-ae04-87c3deb44f03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=510f193078374bf3b129d669d4cd5866, user_id=214e86037f3c4d47bac76a8e0ae76e0f, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:04.354167 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b408bc42-65e1-4b7c-ae04-87c3deb44f03 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:04.371096 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1046/2112] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:58:04 2022] PUT /identity/v3/projects/510f193078374bf3b129d669d4cd5866/users/214e86037f3c4d47bac76a8e0ae76e0f/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:04.381829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98808cfb-4053-4fc7-b462-17e513adcacb None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:04.382150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98808cfb-4053-4fc7-b462-17e513adcacb None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:04.382433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98808cfb-4053-4fc7-b462-17e513adcacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:04.410898 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-98808cfb-4053-4fc7-b462-17e513adcacb None None] MFA Rules not processed for user `214e86037f3c4d47bac76a8e0ae76e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:04.412107 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-98808cfb-4053-4fc7-b462-17e513adcacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:04.475591 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1067/2113] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:04 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:04.480254 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408bc42-65e1-4b7c-ae04-87c3deb44f03 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:04.541377 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408bc42-65e1-4b7c-ae04-87c3deb44f03 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:04.545997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54845a-9b39-4391-b7d7-04887f552374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:04.546668 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54845a-9b39-4391-b7d7-04887f552374 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:04.546964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54845a-9b39-4391-b7d7-04887f552374 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:04.547221 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54845a-9b39-4391-b7d7-04887f552374 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:04.547904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b54845a-9b39-4391-b7d7-04887f552374 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:04.549819 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b54845a-9b39-4391-b7d7-04887f552374 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:04.579152 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1047/2114] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:58:04 2022] POST /identity/v3/projects => generated 367 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:04.580683 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98808cfb-4053-4fc7-b462-17e513adcacb None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:04.595209 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98808cfb-4053-4fc7-b462-17e513adcacb None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:04.621997 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:04.659505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e643d0-7312-4644-9a77-598324250c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:04.660222 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e643d0-7312-4644-9a77-598324250c29 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:04.660511 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e643d0-7312-4644-9a77-598324250c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:04.660770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e643d0-7312-4644-9a77-598324250c29 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:04.661515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e643d0-7312-4644-9a77-598324250c29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:04.663473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e643d0-7312-4644-9a77-598324250c29 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:04.683371 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1068/2115] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:58:04 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:04.687509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54845a-9b39-4391-b7d7-04887f552374 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:04.698613 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54845a-9b39-4391-b7d7-04887f552374 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:04.705882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630eb19a-f5f7-45e6-99be-2f59da1764eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:04.706549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630eb19a-f5f7-45e6-99be-2f59da1764eb admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:04.706702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630eb19a-f5f7-45e6-99be-2f59da1764eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:04.707032 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630eb19a-f5f7-45e6-99be-2f59da1764eb admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:04.707828 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630eb19a-f5f7-45e6-99be-2f59da1764eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:04.709564 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630eb19a-f5f7-45e6-99be-2f59da1764eb admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:04.725687 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1048/2116] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:04.727587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e643d0-7312-4644-9a77-598324250c29 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:04.737917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e643d0-7312-4644-9a77-598324250c29 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:04.743457 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0dfb9-c3e3-4d3d-ba0c-1ff1a34eb995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:04.744381 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0dfb9-c3e3-4d3d-ba0c-1ff1a34eb995 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:04.745438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0dfb9-c3e3-4d3d-ba0c-1ff1a34eb995 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:04.745742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0dfb9-c3e3-4d3d-ba0c-1ff1a34eb995 admin admin] PATH_INFO: `/v3/projects/0ee831df3fec40448dee0b91448675ae/users/60289f5ae0034c49b950c1c115d5d6c2/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:04.761635 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0dfb9-c3e3-4d3d-ba0c-1ff1a34eb995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ee831df3fec40448dee0b91448675ae, user_id=60289f5ae0034c49b950c1c115d5d6c2, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:04.764350 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0dfb9-c3e3-4d3d-ba0c-1ff1a34eb995 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:04.778388 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1069/2117] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:58:04 2022] PUT /identity/v3/projects/0ee831df3fec40448dee0b91448675ae/users/60289f5ae0034c49b950c1c115d5d6c2/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:04.782559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630eb19a-f5f7-45e6-99be-2f59da1764eb admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:04.793671 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630eb19a-f5f7-45e6-99be-2f59da1764eb admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:04.860394 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf8fcf-330f-49b4-b73f-9dda6ac0ce27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:04.861218 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf8fcf-330f-49b4-b73f-9dda6ac0ce27 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:04.861502 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf8fcf-330f-49b4-b73f-9dda6ac0ce27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:04.861769 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf8fcf-330f-49b4-b73f-9dda6ac0ce27 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:04.862710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf8fcf-330f-49b4-b73f-9dda6ac0ce27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:04.864944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf8fcf-330f-49b4-b73f-9dda6ac0ce27 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:04.876851 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1049/2118] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:04 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:04.880940 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0dfb9-c3e3-4d3d-ba0c-1ff1a34eb995 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:04.891930 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0dfb9-c3e3-4d3d-ba0c-1ff1a34eb995 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:04.898140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413689fd-b257-4349-86cb-a1adb70cf15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:04.899782 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413689fd-b257-4349-86cb-a1adb70cf15a admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:04.900620 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413689fd-b257-4349-86cb-a1adb70cf15a admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:04.900906 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413689fd-b257-4349-86cb-a1adb70cf15a admin admin] PATH_INFO: `/v3/projects/0ee831df3fec40448dee0b91448675ae/users/60289f5ae0034c49b950c1c115d5d6c2/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:04.903368 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413689fd-b257-4349-86cb-a1adb70cf15a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ee831df3fec40448dee0b91448675ae, user_id=60289f5ae0034c49b950c1c115d5d6c2, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:04.906970 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413689fd-b257-4349-86cb-a1adb70cf15a admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:04.924817 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1070/2119] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:58:04 2022] PUT /identity/v3/projects/0ee831df3fec40448dee0b91448675ae/users/60289f5ae0034c49b950c1c115d5d6c2/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:04.936680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed745eb-2463-4409-b361-5c7d6fc38028 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:04.936969 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed745eb-2463-4409-b361-5c7d6fc38028 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:04.937285 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed745eb-2463-4409-b361-5c7d6fc38028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:04.964085 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-3ed745eb-2463-4409-b361-5c7d6fc38028 None None] MFA Rules not processed for user `60289f5ae0034c49b950c1c115d5d6c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:04.965315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-3ed745eb-2463-4409-b361-5c7d6fc38028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:04.982076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:05.036939 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1050/2120] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:04 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:05.048726 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413689fd-b257-4349-86cb-a1adb70cf15a admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:05.050717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-413689fd-b257-4349-86cb-a1adb70cf15a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:05.109250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413689fd-b257-4349-86cb-a1adb70cf15a admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:05.112998 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eed2ee-4b7f-412f-b521-be9624fe26c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:05.113643 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eed2ee-4b7f-412f-b521-be9624fe26c3 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:05.113923 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eed2ee-4b7f-412f-b521-be9624fe26c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:05.114233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eed2ee-4b7f-412f-b521-be9624fe26c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:05.125052 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eed2ee-4b7f-412f-b521-be9624fe26c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:05.126569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eed2ee-4b7f-412f-b521-be9624fe26c3 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:05.136870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1071/2121] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:05 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:05.185803 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed745eb-2463-4409-b361-5c7d6fc38028 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:05.187306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-3ed745eb-2463-4409-b361-5c7d6fc38028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:05.199857 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed745eb-2463-4409-b361-5c7d6fc38028 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:05.203492 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56b885-ea62-4270-a358-95bf0a94376b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:05.204133 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56b885-ea62-4270-a358-95bf0a94376b service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:05.204418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56b885-ea62-4270-a358-95bf0a94376b service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:05.204684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56b885-ea62-4270-a358-95bf0a94376b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:05.273986 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56b885-ea62-4270-a358-95bf0a94376b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:05.276208 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56b885-ea62-4270-a358-95bf0a94376b service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:05.286423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1051/2122] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:05 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:05.593461 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eed2ee-4b7f-412f-b521-be9624fe26c3 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:05.595079 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-46eed2ee-4b7f-412f-b521-be9624fe26c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:05.682264 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eed2ee-4b7f-412f-b521-be9624fe26c3 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:05.687031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8fcf0-2e1a-473b-b380-557fafb96a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:05.687963 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8fcf0-2e1a-473b-b380-557fafb96a65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:05.688311 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8fcf0-2e1a-473b-b380-557fafb96a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:05.688592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8fcf0-2e1a-473b-b380-557fafb96a65 admin admin] PATH_INFO: `/v3/users/87141436bd7745dba48f9eae60994149` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:05.689799 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8fcf0-2e1a-473b-b380-557fafb96a65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87141436bd7745dba48f9eae60994149)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:05.691854 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8fcf0-2e1a-473b-b380-557fafb96a65 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:05.716734 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:05.740646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-61f8fcf0-2e1a-473b-b380-557fafb96a65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87141436bd7745dba48f9eae60994149'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:05.753052 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-61f8fcf0-2e1a-473b-b380-557fafb96a65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87141436bd7745dba48f9eae60994149'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:05.762313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-61f8fcf0-2e1a-473b-b380-557fafb96a65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87141436bd7745dba48f9eae60994149'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:05.762990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-61f8fcf0-2e1a-473b-b380-557fafb96a65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87141436bd7745dba48f9eae60994149'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:05.763603 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-61f8fcf0-2e1a-473b-b380-557fafb96a65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87141436bd7745dba48f9eae60994149'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:05.778169 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1072/2123] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:58:05 2022] DELETE /identity/v3/users/87141436bd7745dba48f9eae60994149 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:05.783000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54724a-1166-48ca-8847-3c68ba939049 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:05.783760 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54724a-1166-48ca-8847-3c68ba939049 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:05.783915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54724a-1166-48ca-8847-3c68ba939049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:05.801751 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-2d54724a-1166-48ca-8847-3c68ba939049 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:05.803008 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-2d54724a-1166-48ca-8847-3c68ba939049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:05.864905 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1052/2124] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:58:05.875906 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8fcf0-2e1a-473b-b380-557fafb96a65 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:05.877409 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-61f8fcf0-2e1a-473b-b380-557fafb96a65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:05.940959 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8fcf0-2e1a-473b-b380-557fafb96a65 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:05.947156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6951c-a33e-4a12-adef-65c9ee56ddf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:05.947922 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6951c-a33e-4a12-adef-65c9ee56ddf0 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:05.948220 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6951c-a33e-4a12-adef-65c9ee56ddf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:05.948601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6951c-a33e-4a12-adef-65c9ee56ddf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:05.965567 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6951c-a33e-4a12-adef-65c9ee56ddf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:05.968284 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6951c-a33e-4a12-adef-65c9ee56ddf0 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:05.984415 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1073/2125] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:06.101911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54724a-1166-48ca-8847-3c68ba939049 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:06.104420 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-2d54724a-1166-48ca-8847-3c68ba939049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:06.120765 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54724a-1166-48ca-8847-3c68ba939049 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:06.127428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b9e59-02f5-4cac-ba39-cf300821042a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:06.129028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b9e59-02f5-4cac-ba39-cf300821042a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:06.129450 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b9e59-02f5-4cac-ba39-cf300821042a admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:06.130058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b9e59-02f5-4cac-ba39-cf300821042a admin admin] PATH_INFO: `/v3/users/2753bb5222c4432eb10757771b2020fe` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:06.131827 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0b9e59-02f5-4cac-ba39-cf300821042a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2753bb5222c4432eb10757771b2020fe)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:06.135729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0b9e59-02f5-4cac-ba39-cf300821042a admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:06.193917 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-cb0b9e59-02f5-4cac-ba39-cf300821042a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2753bb5222c4432eb10757771b2020fe'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.195267 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:06.211376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-cb0b9e59-02f5-4cac-ba39-cf300821042a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2753bb5222c4432eb10757771b2020fe'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.220962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-cb0b9e59-02f5-4cac-ba39-cf300821042a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2753bb5222c4432eb10757771b2020fe'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.221596 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-cb0b9e59-02f5-4cac-ba39-cf300821042a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2753bb5222c4432eb10757771b2020fe'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.222855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-cb0b9e59-02f5-4cac-ba39-cf300821042a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2753bb5222c4432eb10757771b2020fe'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.237325 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1053/2126] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:58:06 2022] DELETE /identity/v3/users/2753bb5222c4432eb10757771b2020fe => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:58:06.241870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9b209-7480-437b-bf1a-e6f4d8e8f9c0 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:06.242219 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9b209-7480-437b-bf1a-e6f4d8e8f9c0 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:06.242503 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9b209-7480-437b-bf1a-e6f4d8e8f9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:06.262684 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-41b9b209-7480-437b-bf1a-e6f4d8e8f9c0 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:06.263970 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-41b9b209-7480-437b-bf1a-e6f4d8e8f9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:06.303736 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b9e59-02f5-4cac-ba39-cf300821042a admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:06.305395 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-cb0b9e59-02f5-4cac-ba39-cf300821042a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:06.376262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1074/2127] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:06.386272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9b209-7480-437b-bf1a-e6f4d8e8f9c0 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:06.387683 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-41b9b209-7480-437b-bf1a-e6f4d8e8f9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:06.424600 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b9e59-02f5-4cac-ba39-cf300821042a admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:06.429668 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fd16e-bcaf-403d-aa57-bfefc9ab85e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:06.430711 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fd16e-bcaf-403d-aa57-bfefc9ab85e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:06.431056 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fd16e-bcaf-403d-aa57-bfefc9ab85e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:06.431337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fd16e-bcaf-403d-aa57-bfefc9ab85e1 admin admin] PATH_INFO: `/v3/projects/484e089945844e1c814e75f11ec9001d` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:06.432538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3fd16e-bcaf-403d-aa57-bfefc9ab85e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=484e089945844e1c814e75f11ec9001d)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:06.434495 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3fd16e-bcaf-403d-aa57-bfefc9ab85e1 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:06.457280 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:06.473393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9b209-7480-437b-bf1a-e6f4d8e8f9c0 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:06.490945 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-8c3fd16e-bcaf-403d-aa57-bfefc9ab85e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '484e089945844e1c814e75f11ec9001d', 'request_id': 'req-8c3fd16e-bcaf-403d-aa57-bfefc9ab85e1'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.498071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-8c3fd16e-bcaf-403d-aa57-bfefc9ab85e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '484e089945844e1c814e75f11ec9001d', 'request_id': 'req-8c3fd16e-bcaf-403d-aa57-bfefc9ab85e1'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.505466 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-8c3fd16e-bcaf-403d-aa57-bfefc9ab85e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '484e089945844e1c814e75f11ec9001d', 'request_id': 'req-8c3fd16e-bcaf-403d-aa57-bfefc9ab85e1'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.519707 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-8c3fd16e-bcaf-403d-aa57-bfefc9ab85e1 admin admin] The token cache is being invalidate because project 484e089945844e1c814e75f11ec9001d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:06.519996 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-8c3fd16e-bcaf-403d-aa57-bfefc9ab85e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.520657 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-8c3fd16e-bcaf-403d-aa57-bfefc9ab85e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.522410 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1054/2128] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:58:06 2022] DELETE /identity/v3/projects/484e089945844e1c814e75f11ec9001d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:58:06.527281 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fd16e-bcaf-403d-aa57-bfefc9ab85e1 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:06.529142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-8c3fd16e-bcaf-403d-aa57-bfefc9ab85e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:06.536279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580e1cd-ae43-4d2d-a763-9e369b371e1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:06.536934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580e1cd-ae43-4d2d-a763-9e369b371e1e service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:06.537306 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580e1cd-ae43-4d2d-a763-9e369b371e1e service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:06.537652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580e1cd-ae43-4d2d-a763-9e369b371e1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:06.539352 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e580e1cd-ae43-4d2d-a763-9e369b371e1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:06.597362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:06.605149 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fd16e-bcaf-403d-aa57-bfefc9ab85e1 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:06.613358 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f829e-46fe-4877-9e6d-855feab293fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:06.614385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f829e-46fe-4877-9e6d-855feab293fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:06.614849 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f829e-46fe-4877-9e6d-855feab293fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:06.615336 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f829e-46fe-4877-9e6d-855feab293fb admin admin] PATH_INFO: `/v3/users/2941dca04fea4633b22a0c503fddb6ec` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:06.616815 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f829e-46fe-4877-9e6d-855feab293fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2941dca04fea4633b22a0c503fddb6ec)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:06.618882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f829e-46fe-4877-9e6d-855feab293fb admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:06.625918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e580e1cd-ae43-4d2d-a763-9e369b371e1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:06.627763 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e580e1cd-ae43-4d2d-a763-9e369b371e1e service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:06.648946 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1075/2129] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:58:06.676700 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d33f829e-46fe-4877-9e6d-855feab293fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2941dca04fea4633b22a0c503fddb6ec'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.699391 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d33f829e-46fe-4877-9e6d-855feab293fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2941dca04fea4633b22a0c503fddb6ec'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.711228 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d33f829e-46fe-4877-9e6d-855feab293fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2941dca04fea4633b22a0c503fddb6ec'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.712699 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d33f829e-46fe-4877-9e6d-855feab293fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2941dca04fea4633b22a0c503fddb6ec'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.715877 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d33f829e-46fe-4877-9e6d-855feab293fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2941dca04fea4633b22a0c503fddb6ec'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:06.748756 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1055/2130] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:58:06 2022] DELETE /identity/v3/users/2941dca04fea4633b22a0c503fddb6ec => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:58:07.011545 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580e1cd-ae43-4d2d-a763-9e369b371e1e service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:07.014371 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e580e1cd-ae43-4d2d-a763-9e369b371e1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:07.042900 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580e1cd-ae43-4d2d-a763-9e369b371e1e service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:07.107724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f829e-46fe-4877-9e6d-855feab293fb admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:07.111553 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d33f829e-46fe-4877-9e6d-855feab293fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:07.139272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02957e54-3a74-4e6c-8591-207f828e992d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:07.140416 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02957e54-3a74-4e6c-8591-207f828e992d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:07.140959 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02957e54-3a74-4e6c-8591-207f828e992d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:07.141746 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02957e54-3a74-4e6c-8591-207f828e992d admin admin] PATH_INFO: `/v3/projects/597727823ef34e0fa21c7e74148e30eb` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:07.144834 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02957e54-3a74-4e6c-8591-207f828e992d admin admin] RBAC: Authorizing `identity:delete_project(project_id=597727823ef34e0fa21c7e74148e30eb)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:07.147711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02957e54-3a74-4e6c-8591-207f828e992d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:07.217205 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-02957e54-3a74-4e6c-8591-207f828e992d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '597727823ef34e0fa21c7e74148e30eb', 'request_id': 'req-02957e54-3a74-4e6c-8591-207f828e992d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:07.219474 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:07.229571 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-02957e54-3a74-4e6c-8591-207f828e992d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '597727823ef34e0fa21c7e74148e30eb', 'request_id': 'req-02957e54-3a74-4e6c-8591-207f828e992d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:07.230204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f829e-46fe-4877-9e6d-855feab293fb admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:07.237040 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-02957e54-3a74-4e6c-8591-207f828e992d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '597727823ef34e0fa21c7e74148e30eb', 'request_id': 'req-02957e54-3a74-4e6c-8591-207f828e992d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:07.252142 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-02957e54-3a74-4e6c-8591-207f828e992d admin admin] The token cache is being invalidate because project 597727823ef34e0fa21c7e74148e30eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:07.253130 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-02957e54-3a74-4e6c-8591-207f828e992d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:07.253973 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-02957e54-3a74-4e6c-8591-207f828e992d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:07.256159 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1076/2131] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:58:07 2022] DELETE /identity/v3/projects/597727823ef34e0fa21c7e74148e30eb => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:07.261463 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b9850-ae10-436d-8b6f-af55b8215ab6 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:07.261754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b9850-ae10-436d-8b6f-af55b8215ab6 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:07.262216 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b9850-ae10-436d-8b6f-af55b8215ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:07.281119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-460b9850-ae10-436d-8b6f-af55b8215ab6 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:07.282484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-460b9850-ae10-436d-8b6f-af55b8215ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:07.294669 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd7805-9fcc-4696-a7dd-7069cbd34e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:07.295552 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd7805-9fcc-4696-a7dd-7069cbd34e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:07.295833 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd7805-9fcc-4696-a7dd-7069cbd34e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:07.296113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd7805-9fcc-4696-a7dd-7069cbd34e97 admin admin] PATH_INFO: `/v3/projects/ed0fa65aba4b48a6a43e72bbc0bab060` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:07.297372 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd7805-9fcc-4696-a7dd-7069cbd34e97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed0fa65aba4b48a6a43e72bbc0bab060)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:07.299366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd7805-9fcc-4696-a7dd-7069cbd34e97 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:07.326265 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:07.350996 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1077/2132] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:58:07.351212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-79cd7805-9fcc-4696-a7dd-7069cbd34e97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed0fa65aba4b48a6a43e72bbc0bab060', 'request_id': 'req-79cd7805-9fcc-4696-a7dd-7069cbd34e97'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:07.362107 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046c056-a6f3-45cf-9ddc-f008b8e60926 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:07.362402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046c056-a6f3-45cf-9ddc-f008b8e60926 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:07.362680 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046c056-a6f3-45cf-9ddc-f008b8e60926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:07.363809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-79cd7805-9fcc-4696-a7dd-7069cbd34e97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed0fa65aba4b48a6a43e72bbc0bab060', 'request_id': 'req-79cd7805-9fcc-4696-a7dd-7069cbd34e97'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:07.372776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-79cd7805-9fcc-4696-a7dd-7069cbd34e97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed0fa65aba4b48a6a43e72bbc0bab060', 'request_id': 'req-79cd7805-9fcc-4696-a7dd-7069cbd34e97'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:07.386948 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-79cd7805-9fcc-4696-a7dd-7069cbd34e97 admin admin] The token cache is being invalidate because project ed0fa65aba4b48a6a43e72bbc0bab060 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:07.387362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-79cd7805-9fcc-4696-a7dd-7069cbd34e97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:07.387477 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-5046c056-a6f3-45cf-9ddc-f008b8e60926 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:07.388014 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-79cd7805-9fcc-4696-a7dd-7069cbd34e97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:07.388798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5046c056-a6f3-45cf-9ddc-f008b8e60926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:07.389601 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1056/2133] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:58:07 2022] DELETE /identity/v3/projects/ed0fa65aba4b48a6a43e72bbc0bab060 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:07.398175 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03251b18-6e29-42bc-8cc4-06bd745fda5a None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:07.398175 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03251b18-6e29-42bc-8cc4-06bd745fda5a None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:07.398175 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03251b18-6e29-42bc-8cc4-06bd745fda5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:07.421591 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-03251b18-6e29-42bc-8cc4-06bd745fda5a None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:07.423347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-03251b18-6e29-42bc-8cc4-06bd745fda5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:07.459856 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1078/2134] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:58:07.464088 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046c056-a6f3-45cf-9ddc-f008b8e60926 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:07.473240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046c056-a6f3-45cf-9ddc-f008b8e60926 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:07.478943 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98b7c6-ae03-4035-bc42-d17d3b02dbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:07.479628 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98b7c6-ae03-4035-bc42-d17d3b02dbc0 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:07.479919 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98b7c6-ae03-4035-bc42-d17d3b02dbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:07.480201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98b7c6-ae03-4035-bc42-d17d3b02dbc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:07.480941 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98b7c6-ae03-4035-bc42-d17d3b02dbc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:58:07.481259 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98b7c6-ae03-4035-bc42-d17d3b02dbc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:07.482917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98b7c6-ae03-4035-bc42-d17d3b02dbc0 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:07.496084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1057/2135] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:58:07.496732 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1079/2136] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:58:07.504446 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3896b7-6932-4f75-b7e3-6241d8553ff3 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:07.505007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3896b7-6932-4f75-b7e3-6241d8553ff3 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:07.505359 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3896b7-6932-4f75-b7e3-6241d8553ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:07.506900 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98b7c6-ae03-4035-bc42-d17d3b02dbc0 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:07.517574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98b7c6-ae03-4035-bc42-d17d3b02dbc0 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:07.522428 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c6bcb-4a82-4bf0-91eb-9f8217a45234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:07.523086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c6bcb-4a82-4bf0-91eb-9f8217a45234 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:07.523370 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c6bcb-4a82-4bf0-91eb-9f8217a45234 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:07.523645 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c6bcb-4a82-4bf0-91eb-9f8217a45234 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:07.524394 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c6bcb-4a82-4bf0-91eb-9f8217a45234 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:07.526057 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-9b3896b7-6932-4f75-b7e3-6241d8553ff3 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:07.526349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c6bcb-4a82-4bf0-91eb-9f8217a45234 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:07.527354 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-9b3896b7-6932-4f75-b7e3-6241d8553ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:07.540984 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1058/2137] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:58:07.548224 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3896b7-6932-4f75-b7e3-6241d8553ff3 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:07.560219 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3896b7-6932-4f75-b7e3-6241d8553ff3 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:07.565099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b8378-7e3a-413c-8b85-73b0ec4d742c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:07.565676 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1080/2138] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:58:07 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:58:07.565804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b8378-7e3a-413c-8b85-73b0ec4d742c admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:07.566044 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b8378-7e3a-413c-8b85-73b0ec4d742c admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:07.566325 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b8378-7e3a-413c-8b85-73b0ec4d742c admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:07.567048 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b8378-7e3a-413c-8b85-73b0ec4d742c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:58:07.567350 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b8378-7e3a-413c-8b85-73b0ec4d742c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:07.568937 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b8378-7e3a-413c-8b85-73b0ec4d742c admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:07.569620 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c6bcb-4a82-4bf0-91eb-9f8217a45234 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:07.583804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1059/2139] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:58:07.587987 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b8378-7e3a-413c-8b85-73b0ec4d742c admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:07.601090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b8378-7e3a-413c-8b85-73b0ec4d742c admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:07.631031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:07.651968 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c6bcb-4a82-4bf0-91eb-9f8217a45234 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:07.657514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b97d12-718e-414c-a314-1c2f0ae38dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:07.658306 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b97d12-718e-414c-a314-1c2f0ae38dd3 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:07.658592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b97d12-718e-414c-a314-1c2f0ae38dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:07.658873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b97d12-718e-414c-a314-1c2f0ae38dd3 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:07.659746 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b97d12-718e-414c-a314-1c2f0ae38dd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:07.661747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b97d12-718e-414c-a314-1c2f0ae38dd3 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:07.686127 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488b57d-639d-466e-b42b-3429287edeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:07.687280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488b57d-639d-466e-b42b-3429287edeeb admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:07.687621 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488b57d-639d-466e-b42b-3429287edeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:07.687962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488b57d-639d-466e-b42b-3429287edeeb admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:07.688807 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488b57d-639d-466e-b42b-3429287edeeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:07.690644 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1081/2140] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:58:07 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:07.690779 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488b57d-639d-466e-b42b-3429287edeeb admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:07.694450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b97d12-718e-414c-a314-1c2f0ae38dd3 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:07.708054 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b97d12-718e-414c-a314-1c2f0ae38dd3 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:07.717220 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a156e-7a88-4ad6-9580-21a9b1a57ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:07.718186 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a156e-7a88-4ad6-9580-21a9b1a57ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:07.718466 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a156e-7a88-4ad6-9580-21a9b1a57ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:07.718833 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a156e-7a88-4ad6-9580-21a9b1a57ea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:07.719535 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a156e-7a88-4ad6-9580-21a9b1a57ea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:07.721154 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a156e-7a88-4ad6-9580-21a9b1a57ea4 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:07.722694 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1060/2141] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:58:07 2022] POST /identity/v3/projects => generated 377 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:58:07.726497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488b57d-639d-466e-b42b-3429287edeeb admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:07.737140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1082/2142] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:07.739756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a156e-7a88-4ad6-9580-21a9b1a57ea4 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:07.752269 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a156e-7a88-4ad6-9580-21a9b1a57ea4 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:07.755979 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:07.822125 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051edd1a-3d93-47bd-adfb-ddde7ec1bc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:07.822788 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051edd1a-3d93-47bd-adfb-ddde7ec1bc45 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:07.823092 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051edd1a-3d93-47bd-adfb-ddde7ec1bc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:07.823343 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051edd1a-3d93-47bd-adfb-ddde7ec1bc45 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:07.824110 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051edd1a-3d93-47bd-adfb-ddde7ec1bc45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:07.826060 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051edd1a-3d93-47bd-adfb-ddde7ec1bc45 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:07.832898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488b57d-639d-466e-b42b-3429287edeeb admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:07.839137 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b64aa82-1c02-413c-a8b6-68e7d8525bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:07.839647 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1083/2143] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:58:07.839812 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b64aa82-1c02-413c-a8b6-68e7d8525bbe admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:07.840084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b64aa82-1c02-413c-a8b6-68e7d8525bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:07.840419 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b64aa82-1c02-413c-a8b6-68e7d8525bbe admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:07.841110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b64aa82-1c02-413c-a8b6-68e7d8525bbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:07.843295 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b64aa82-1c02-413c-a8b6-68e7d8525bbe admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:07.844533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051edd1a-3d93-47bd-adfb-ddde7ec1bc45 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:07.856938 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051edd1a-3d93-47bd-adfb-ddde7ec1bc45 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:07.861896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941d245-2d01-4e28-8367-81811b8f9f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:07.863037 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941d245-2d01-4e28-8367-81811b8f9f99 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:07.863328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941d245-2d01-4e28-8367-81811b8f9f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:07.863723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941d245-2d01-4e28-8367-81811b8f9f99 admin admin] PATH_INFO: `/v3/projects/56174c2dc6ce472697756899661f4838/users/aef8ca7cd35d451ba228a058892f2da4/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:07.875300 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1061/2144] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:58:07 2022] POST /identity/v3/users => generated 289 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:58:07.877144 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b64aa82-1c02-413c-a8b6-68e7d8525bbe admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:07.882720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0941d245-2d01-4e28-8367-81811b8f9f99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56174c2dc6ce472697756899661f4838, user_id=aef8ca7cd35d451ba228a058892f2da4, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:07.885546 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0941d245-2d01-4e28-8367-81811b8f9f99 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:07.887371 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b64aa82-1c02-413c-a8b6-68e7d8525bbe admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:07.892959 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3372c-64e7-4315-8edb-94030d0e3012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:07.893789 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3372c-64e7-4315-8edb-94030d0e3012 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:07.894261 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3372c-64e7-4315-8edb-94030d0e3012 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:07.894540 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3372c-64e7-4315-8edb-94030d0e3012 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:07.895304 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3372c-64e7-4315-8edb-94030d0e3012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:07.896919 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3372c-64e7-4315-8edb-94030d0e3012 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:07.903208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1084/2145] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:58:07 2022] PUT /identity/v3/projects/56174c2dc6ce472697756899661f4838/users/aef8ca7cd35d451ba228a058892f2da4/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:07.910787 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1062/2146] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:58:07.915527 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941d245-2d01-4e28-8367-81811b8f9f99 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:07.916163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ca07f-3d94-4ae6-9218-b8d3dac5fcfc None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:07.916448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ca07f-3d94-4ae6-9218-b8d3dac5fcfc None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:07.916756 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ca07f-3d94-4ae6-9218-b8d3dac5fcfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:07.947256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-178ca07f-3d94-4ae6-9218-b8d3dac5fcfc None None] MFA Rules not processed for user `aef8ca7cd35d451ba228a058892f2da4`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:07.948490 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-178ca07f-3d94-4ae6-9218-b8d3dac5fcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:07.982599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941d245-2d01-4e28-8367-81811b8f9f99 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:07.987956 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4244094-3131-4d07-87a0-ecb3c82d110e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:07.988980 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4244094-3131-4d07-87a0-ecb3c82d110e admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:07.989378 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4244094-3131-4d07-87a0-ecb3c82d110e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:07.989760 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4244094-3131-4d07-87a0-ecb3c82d110e admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:07.990669 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4244094-3131-4d07-87a0-ecb3c82d110e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:07.992540 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4244094-3131-4d07-87a0-ecb3c82d110e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:08.003319 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1085/2147] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:58:08.007067 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4244094-3131-4d07-87a0-ecb3c82d110e admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:08.018763 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4244094-3131-4d07-87a0-ecb3c82d110e admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:08.023510 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7dcd2-0ed1-4d08-843a-ecc24df87b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:08.023735 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1063/2148] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:58:08.026755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7dcd2-0ed1-4d08-843a-ecc24df87b00 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:08.027099 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7dcd2-0ed1-4d08-843a-ecc24df87b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:08.027379 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7dcd2-0ed1-4d08-843a-ecc24df87b00 admin admin] PATH_INFO: `/v3/projects/6e27eea48a204fb196e5bad0c061924b/users/0d313b61fa24407b846372b796701242/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:08.035588 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ca07f-3d94-4ae6-9218-b8d3dac5fcfc None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:08.036955 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-178ca07f-3d94-4ae6-9218-b8d3dac5fcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:08.044007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7dcd2-0ed1-4d08-843a-ecc24df87b00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e27eea48a204fb196e5bad0c061924b, user_id=0d313b61fa24407b846372b796701242, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:08.046755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7dcd2-0ed1-4d08-843a-ecc24df87b00 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:08.064142 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1086/2149] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:58:08 2022] PUT /identity/v3/projects/6e27eea48a204fb196e5bad0c061924b/users/0d313b61fa24407b846372b796701242/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:58:08.074412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54942e96-8082-42c6-b171-a845a0864e56 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:08.074810 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54942e96-8082-42c6-b171-a845a0864e56 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:08.075164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54942e96-8082-42c6-b171-a845a0864e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:08.093656 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:08.099101 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ca07f-3d94-4ae6-9218-b8d3dac5fcfc None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:08.106507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-54942e96-8082-42c6-b171-a845a0864e56 None None] MFA Rules not processed for user `0d313b61fa24407b846372b796701242`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:08.107745 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-54942e96-8082-42c6-b171-a845a0864e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:08.144471 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:08.160334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb95fb-cecb-4910-b1af-dd6f447adeff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:08.160902 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb95fb-cecb-4910-b1af-dd6f447adeff service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:08.161182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb95fb-cecb-4910-b1af-dd6f447adeff service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:08.161483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb95fb-cecb-4910-b1af-dd6f447adeff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:08.180720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1087/2150] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:08.185183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54942e96-8082-42c6-b171-a845a0864e56 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:08.237835 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbb95fb-cecb-4910-b1af-dd6f447adeff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:08.239470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbb95fb-cecb-4910-b1af-dd6f447adeff service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:08.249884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1064/2151] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:58:08.259145 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54942e96-8082-42c6-b171-a845a0864e56 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:08.267585 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa100f3-983d-4764-bee1-d21a246dbdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:08.268255 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa100f3-983d-4764-bee1-d21a246dbdec admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:08.268534 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa100f3-983d-4764-bee1-d21a246dbdec admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:08.268813 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa100f3-983d-4764-bee1-d21a246dbdec admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:08.269506 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa100f3-983d-4764-bee1-d21a246dbdec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:08.271443 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa100f3-983d-4764-bee1-d21a246dbdec admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:08.310178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1088/2152] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:58:08 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:58:08.312347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb95fb-cecb-4910-b1af-dd6f447adeff service cinder] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:08.329036 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb95fb-cecb-4910-b1af-dd6f447adeff service cinder] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:08.406268 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd68308-0754-4f82-8860-a93b5ca97d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:08.407014 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd68308-0754-4f82-8860-a93b5ca97d02 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:08.407359 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd68308-0754-4f82-8860-a93b5ca97d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:08.407694 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd68308-0754-4f82-8860-a93b5ca97d02 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:08.408569 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd68308-0754-4f82-8860-a93b5ca97d02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:08.410837 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd68308-0754-4f82-8860-a93b5ca97d02 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:08.435779 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1065/2153] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:58:08 2022] POST /identity/v3/users => generated 294 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:08.440880 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa100f3-983d-4764-bee1-d21a246dbdec admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:08.452377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa100f3-983d-4764-bee1-d21a246dbdec admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:08.457591 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d277946-bfcf-4aa4-a7f5-eaa3d4b838b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:08.458174 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d277946-bfcf-4aa4-a7f5-eaa3d4b838b6 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:08.458433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d277946-bfcf-4aa4-a7f5-eaa3d4b838b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:08.458714 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d277946-bfcf-4aa4-a7f5-eaa3d4b838b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:08.459438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d277946-bfcf-4aa4-a7f5-eaa3d4b838b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:08.461049 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d277946-bfcf-4aa4-a7f5-eaa3d4b838b6 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:08.475045 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1089/2154] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:08.479514 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd68308-0754-4f82-8860-a93b5ca97d02 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:08.492414 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd68308-0754-4f82-8860-a93b5ca97d02 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:08.497749 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190395f-ccd9-43e3-88b5-c51dadfd5906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:08.498790 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190395f-ccd9-43e3-88b5-c51dadfd5906 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:08.499069 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190395f-ccd9-43e3-88b5-c51dadfd5906 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:08.499348 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190395f-ccd9-43e3-88b5-c51dadfd5906 admin admin] PATH_INFO: `/v3/projects/72bd84196a454d5787a9a89726df6182/users/61af1910640e45aeb66920c60e64cdbe/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:08.527079 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190395f-ccd9-43e3-88b5-c51dadfd5906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72bd84196a454d5787a9a89726df6182, user_id=61af1910640e45aeb66920c60e64cdbe, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:08.530137 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190395f-ccd9-43e3-88b5-c51dadfd5906 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:08.552294 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1066/2155] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:58:08 2022] PUT /identity/v3/projects/72bd84196a454d5787a9a89726df6182/users/61af1910640e45aeb66920c60e64cdbe/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:08.557236 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d277946-bfcf-4aa4-a7f5-eaa3d4b838b6 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:08.570560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d277946-bfcf-4aa4-a7f5-eaa3d4b838b6 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:08.655337 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d62638-529e-4e3e-813c-a46c1322f0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:08.656021 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d62638-529e-4e3e-813c-a46c1322f0c5 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:08.656303 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d62638-529e-4e3e-813c-a46c1322f0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:08.656575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d62638-529e-4e3e-813c-a46c1322f0c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:08.657299 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d62638-529e-4e3e-813c-a46c1322f0c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:08.658999 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d62638-529e-4e3e-813c-a46c1322f0c5 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:08.670228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1090/2156] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:58:08.674458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190395f-ccd9-43e3-88b5-c51dadfd5906 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:08.685355 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190395f-ccd9-43e3-88b5-c51dadfd5906 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:08.691448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648ba61-6f69-47a6-adac-15da437aabdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:08.692582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648ba61-6f69-47a6-adac-15da437aabdf admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:08.693037 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648ba61-6f69-47a6-adac-15da437aabdf admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:08.693246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648ba61-6f69-47a6-adac-15da437aabdf admin admin] PATH_INFO: `/v3/projects/72bd84196a454d5787a9a89726df6182/users/61af1910640e45aeb66920c60e64cdbe/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:08.695410 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c648ba61-6f69-47a6-adac-15da437aabdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=72bd84196a454d5787a9a89726df6182, user_id=61af1910640e45aeb66920c60e64cdbe, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:08.698764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c648ba61-6f69-47a6-adac-15da437aabdf admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:08.709371 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1067/2157] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:58:08 2022] PUT /identity/v3/projects/72bd84196a454d5787a9a89726df6182/users/61af1910640e45aeb66920c60e64cdbe/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:08.720576 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33f4c1-5ffa-4767-9b00-de3eee777c2a None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:08.720906 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33f4c1-5ffa-4767-9b00-de3eee777c2a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:08.721291 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33f4c1-5ffa-4767-9b00-de3eee777c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:08.743727 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:08.754692 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-de33f4c1-5ffa-4767-9b00-de3eee777c2a None None] MFA Rules not processed for user `61af1910640e45aeb66920c60e64cdbe`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:08.755902 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-de33f4c1-5ffa-4767-9b00-de3eee777c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:08.840293 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1091/2158] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:08 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:58:08.851865 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648ba61-6f69-47a6-adac-15da437aabdf admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:08.932422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:08.935863 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de33f4c1-5ffa-4767-9b00-de3eee777c2a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:08.937114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648ba61-6f69-47a6-adac-15da437aabdf admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:08.937624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-de33f4c1-5ffa-4767-9b00-de3eee777c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:08.943301 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d485d-b8d1-41fc-a482-30b7120b676c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:08.944166 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d485d-b8d1-41fc-a482-30b7120b676c service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:08.944767 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d485d-b8d1-41fc-a482-30b7120b676c service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:08.945344 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d485d-b8d1-41fc-a482-30b7120b676c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:09.005121 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d485d-b8d1-41fc-a482-30b7120b676c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:09.006644 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d485d-b8d1-41fc-a482-30b7120b676c service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:09.016959 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1068/2159] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:09.020324 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de33f4c1-5ffa-4767-9b00-de3eee777c2a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:09.025785 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e80cc7-2076-4aca-b4d8-280cafdc1171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:09.027365 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e80cc7-2076-4aca-b4d8-280cafdc1171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:09.027879 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e80cc7-2076-4aca-b4d8-280cafdc1171 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:09.028392 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e80cc7-2076-4aca-b4d8-280cafdc1171 admin admin] PATH_INFO: `/v3/users/214e86037f3c4d47bac76a8e0ae76e0f` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:09.029989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e80cc7-2076-4aca-b4d8-280cafdc1171 admin admin] RBAC: Authorizing `identity:delete_user(user_id=214e86037f3c4d47bac76a8e0ae76e0f)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:09.031974 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e80cc7-2076-4aca-b4d8-280cafdc1171 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:09.092037 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-b3e80cc7-2076-4aca-b4d8-280cafdc1171 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '214e86037f3c4d47bac76a8e0ae76e0f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:09.105730 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-b3e80cc7-2076-4aca-b4d8-280cafdc1171 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '214e86037f3c4d47bac76a8e0ae76e0f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:09.114495 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-b3e80cc7-2076-4aca-b4d8-280cafdc1171 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '214e86037f3c4d47bac76a8e0ae76e0f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:09.115150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-b3e80cc7-2076-4aca-b4d8-280cafdc1171 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '214e86037f3c4d47bac76a8e0ae76e0f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:09.115886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-b3e80cc7-2076-4aca-b4d8-280cafdc1171 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '214e86037f3c4d47bac76a8e0ae76e0f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:09.134260 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1092/2160] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:58:08 2022] DELETE /identity/v3/users/214e86037f3c4d47bac76a8e0ae76e0f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:58:09.138951 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf3915-6e29-470a-b8e9-52beae67589d None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:09.139235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf3915-6e29-470a-b8e9-52beae67589d None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:09.139514 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf3915-6e29-470a-b8e9-52beae67589d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:09.157100 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-1edf3915-6e29-470a-b8e9-52beae67589d None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:09.158392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-1edf3915-6e29-470a-b8e9-52beae67589d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:09.244618 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1069/2161] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:09.251213 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e80cc7-2076-4aca-b4d8-280cafdc1171 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:09.252679 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b3e80cc7-2076-4aca-b4d8-280cafdc1171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:09.268396 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:09.331934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e80cc7-2076-4aca-b4d8-280cafdc1171 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:09.335991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8198de0-5506-452c-8f33-5ebd4cef1aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:09.337040 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8198de0-5506-452c-8f33-5ebd4cef1aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:09.337436 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8198de0-5506-452c-8f33-5ebd4cef1aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:09.337678 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8198de0-5506-452c-8f33-5ebd4cef1aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:09.348984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8198de0-5506-452c-8f33-5ebd4cef1aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:09.350542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8198de0-5506-452c-8f33-5ebd4cef1aa0 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:09.365183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1093/2162] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:09.646969 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf3915-6e29-470a-b8e9-52beae67589d None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:09.648392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-1edf3915-6e29-470a-b8e9-52beae67589d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:09.660414 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf3915-6e29-470a-b8e9-52beae67589d None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:09.666155 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6d0c7-7f43-4741-8d71-66b01c077ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:09.667046 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6d0c7-7f43-4741-8d71-66b01c077ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:09.677939 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6d0c7-7f43-4741-8d71-66b01c077ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:09.677939 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6d0c7-7f43-4741-8d71-66b01c077ea4 admin admin] PATH_INFO: `/v3/projects/510f193078374bf3b129d669d4cd5866` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:09.677939 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6d0c7-7f43-4741-8d71-66b01c077ea4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=510f193078374bf3b129d669d4cd5866)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:09.677939 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6d0c7-7f43-4741-8d71-66b01c077ea4 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:09.685171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35102f64-5025-4f57-9aaa-6fb38fe165d7 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:09.685463 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35102f64-5025-4f57-9aaa-6fb38fe165d7 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:09.685740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35102f64-5025-4f57-9aaa-6fb38fe165d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:09.703429 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-35102f64-5025-4f57-9aaa-6fb38fe165d7 None None] MFA Rules not processed for user `aef8ca7cd35d451ba228a058892f2da4`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:09.704779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-35102f64-5025-4f57-9aaa-6fb38fe165d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:09.720287 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-59f6d0c7-7f43-4741-8d71-66b01c077ea4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '510f193078374bf3b129d669d4cd5866', 'request_id': 'req-59f6d0c7-7f43-4741-8d71-66b01c077ea4'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:09.728127 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-59f6d0c7-7f43-4741-8d71-66b01c077ea4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '510f193078374bf3b129d669d4cd5866', 'request_id': 'req-59f6d0c7-7f43-4741-8d71-66b01c077ea4'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:09.735970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-59f6d0c7-7f43-4741-8d71-66b01c077ea4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '510f193078374bf3b129d669d4cd5866', 'request_id': 'req-59f6d0c7-7f43-4741-8d71-66b01c077ea4'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:09.750426 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-59f6d0c7-7f43-4741-8d71-66b01c077ea4 admin admin] The token cache is being invalidate because project 510f193078374bf3b129d669d4cd5866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:09.750758 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-59f6d0c7-7f43-4741-8d71-66b01c077ea4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:09.751386 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-59f6d0c7-7f43-4741-8d71-66b01c077ea4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:09.752968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1070/2163] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:58:09 2022] DELETE /identity/v3/projects/510f193078374bf3b129d669d4cd5866 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:09.756891 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6d0c7-7f43-4741-8d71-66b01c077ea4 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:09.758300 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-59f6d0c7-7f43-4741-8d71-66b01c077ea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:09.763947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1094/2164] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:58:09.795704 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:09.830711 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6d0c7-7f43-4741-8d71-66b01c077ea4 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:09.835562 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6245457a-45bb-43c5-b39d-d8b22f9d75fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:09.836448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245457a-45bb-43c5-b39d-d8b22f9d75fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:09.836729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245457a-45bb-43c5-b39d-d8b22f9d75fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:09.837003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245457a-45bb-43c5-b39d-d8b22f9d75fd admin admin] PATH_INFO: `/v3/users/60289f5ae0034c49b950c1c115d5d6c2` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:09.838351 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6245457a-45bb-43c5-b39d-d8b22f9d75fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=60289f5ae0034c49b950c1c115d5d6c2)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:09.840281 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6245457a-45bb-43c5-b39d-d8b22f9d75fd admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:09.891004 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6245457a-45bb-43c5-b39d-d8b22f9d75fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60289f5ae0034c49b950c1c115d5d6c2'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:09.903495 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6245457a-45bb-43c5-b39d-d8b22f9d75fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60289f5ae0034c49b950c1c115d5d6c2'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:09.911042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6245457a-45bb-43c5-b39d-d8b22f9d75fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60289f5ae0034c49b950c1c115d5d6c2'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:09.911664 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6245457a-45bb-43c5-b39d-d8b22f9d75fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60289f5ae0034c49b950c1c115d5d6c2'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:09.912279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6245457a-45bb-43c5-b39d-d8b22f9d75fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60289f5ae0034c49b950c1c115d5d6c2'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:09.927687 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1071/2165] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:58:09 2022] DELETE /identity/v3/users/60289f5ae0034c49b950c1c115d5d6c2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:58:10.245353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35102f64-5025-4f57-9aaa-6fb38fe165d7 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:10.260275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35102f64-5025-4f57-9aaa-6fb38fe165d7 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:10.334148 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4fdce-7be4-47c1-af19-d6c95eb6a04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:10.335226 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4fdce-7be4-47c1-af19-d6c95eb6a04b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:10.335585 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4fdce-7be4-47c1-af19-d6c95eb6a04b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:10.336004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4fdce-7be4-47c1-af19-d6c95eb6a04b admin admin] PATH_INFO: `/v3/projects/0ee831df3fec40448dee0b91448675ae` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:10.338038 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4fdce-7be4-47c1-af19-d6c95eb6a04b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ee831df3fec40448dee0b91448675ae)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:10.340150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4fdce-7be4-47c1-af19-d6c95eb6a04b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:10.393986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-32e4fdce-7be4-47c1-af19-d6c95eb6a04b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ee831df3fec40448dee0b91448675ae', 'request_id': 'req-32e4fdce-7be4-47c1-af19-d6c95eb6a04b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:10.401528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-32e4fdce-7be4-47c1-af19-d6c95eb6a04b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ee831df3fec40448dee0b91448675ae', 'request_id': 'req-32e4fdce-7be4-47c1-af19-d6c95eb6a04b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:10.418341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-32e4fdce-7be4-47c1-af19-d6c95eb6a04b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ee831df3fec40448dee0b91448675ae', 'request_id': 'req-32e4fdce-7be4-47c1-af19-d6c95eb6a04b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:10.418341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:10.423261 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-32e4fdce-7be4-47c1-af19-d6c95eb6a04b admin admin] The token cache is being invalidate because project 0ee831df3fec40448dee0b91448675ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:10.423625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-32e4fdce-7be4-47c1-af19-d6c95eb6a04b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:10.424331 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-32e4fdce-7be4-47c1-af19-d6c95eb6a04b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:10.426215 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1095/2166] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:58:10 2022] DELETE /identity/v3/projects/0ee831df3fec40448dee0b91448675ae => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:10.431328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b96232d-6695-43a9-87f1-949dc9adf54d None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:10.431833 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b96232d-6695-43a9-87f1-949dc9adf54d None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:10.431936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b96232d-6695-43a9-87f1-949dc9adf54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:10.457485 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-6b96232d-6695-43a9-87f1-949dc9adf54d None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:10.461103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-6b96232d-6695-43a9-87f1-949dc9adf54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:10.538477 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1072/2167] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:10.546376 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d41c3-3f8d-49d2-b25c-d0d17695830c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:10.546655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d41c3-3f8d-49d2-b25c-d0d17695830c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:10.546941 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d41c3-3f8d-49d2-b25c-d0d17695830c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:10.566289 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-811d41c3-3f8d-49d2-b25c-d0d17695830c None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:10.567744 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-811d41c3-3f8d-49d2-b25c-d0d17695830c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:10.580545 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1096/2168] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:10.584845 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b96232d-6695-43a9-87f1-949dc9adf54d None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:10.602507 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b96232d-6695-43a9-87f1-949dc9adf54d None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:10.609897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75b536-d29e-4f46-8f04-d543055d425d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:10.610814 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75b536-d29e-4f46-8f04-d543055d425d admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:10.611127 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75b536-d29e-4f46-8f04-d543055d425d admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:10.611380 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75b536-d29e-4f46-8f04-d543055d425d admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:10.612119 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75b536-d29e-4f46-8f04-d543055d425d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:58:10.612423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75b536-d29e-4f46-8f04-d543055d425d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:10.614064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75b536-d29e-4f46-8f04-d543055d425d admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:10.626780 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1073/2169] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:58:10.634992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d41c3-3f8d-49d2-b25c-d0d17695830c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:10.644894 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d41c3-3f8d-49d2-b25c-d0d17695830c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:10.650697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49684be-3aa8-42b6-8b8c-43e05bfdbcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:10.651485 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49684be-3aa8-42b6-8b8c-43e05bfdbcd1 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:10.651846 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49684be-3aa8-42b6-8b8c-43e05bfdbcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:10.652159 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49684be-3aa8-42b6-8b8c-43e05bfdbcd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:10.652891 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49684be-3aa8-42b6-8b8c-43e05bfdbcd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:10.654980 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49684be-3aa8-42b6-8b8c-43e05bfdbcd1 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:10.680851 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1097/2170] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:58:10 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:10.685165 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75b536-d29e-4f46-8f04-d543055d425d admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:10.695210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75b536-d29e-4f46-8f04-d543055d425d admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:10.725750 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:10.755992 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9a1f5-28b6-45e9-b07e-167b3f9c745c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:10.756661 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9a1f5-28b6-45e9-b07e-167b3f9c745c admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:10.756942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9a1f5-28b6-45e9-b07e-167b3f9c745c admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:10.757213 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9a1f5-28b6-45e9-b07e-167b3f9c745c admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:10.757900 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9a1f5-28b6-45e9-b07e-167b3f9c745c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:10.760446 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9a1f5-28b6-45e9-b07e-167b3f9c745c admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:10.781231 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1074/2171] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:58:10 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:10.785955 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49684be-3aa8-42b6-8b8c-43e05bfdbcd1 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:10.796865 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49684be-3aa8-42b6-8b8c-43e05bfdbcd1 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:10.801742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d5dff-ff01-48ae-9182-de3d012aa421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:10.802471 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d5dff-ff01-48ae-9182-de3d012aa421 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:10.802760 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d5dff-ff01-48ae-9182-de3d012aa421 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:10.803043 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d5dff-ff01-48ae-9182-de3d012aa421 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:10.803921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d5dff-ff01-48ae-9182-de3d012aa421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:10.806109 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d5dff-ff01-48ae-9182-de3d012aa421 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:10.817617 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1098/2172] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:10.822364 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9a1f5-28b6-45e9-b07e-167b3f9c745c admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:10.832282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9a1f5-28b6-45e9-b07e-167b3f9c745c admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:10.836913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc653ce1-a17d-446a-ae2e-c526b88507f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:10.837557 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc653ce1-a17d-446a-ae2e-c526b88507f0 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:10.837844 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc653ce1-a17d-446a-ae2e-c526b88507f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:10.838138 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc653ce1-a17d-446a-ae2e-c526b88507f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:10.838859 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc653ce1-a17d-446a-ae2e-c526b88507f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:10.840431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc653ce1-a17d-446a-ae2e-c526b88507f0 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:10.850268 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1075/2173] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:58:10.854495 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d5dff-ff01-48ae-9182-de3d012aa421 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:10.864661 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d5dff-ff01-48ae-9182-de3d012aa421 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:10.869855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a7a8d-ae4e-4c39-a3de-16c5a7f3e9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:10.870789 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a7a8d-ae4e-4c39-a3de-16c5a7f3e9a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:10.871078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a7a8d-ae4e-4c39-a3de-16c5a7f3e9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:10.871357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a7a8d-ae4e-4c39-a3de-16c5a7f3e9a2 admin admin] PATH_INFO: `/v3/projects/b93a528b4b014972acd9e92c28e345bd/users/c17a6b099ec14068b1e465024f75ed1d/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:10.886671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a7a8d-ae4e-4c39-a3de-16c5a7f3e9a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b93a528b4b014972acd9e92c28e345bd, user_id=c17a6b099ec14068b1e465024f75ed1d, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:10.889363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a7a8d-ae4e-4c39-a3de-16c5a7f3e9a2 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:10.904243 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1099/2174] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:58:10 2022] PUT /identity/v3/projects/b93a528b4b014972acd9e92c28e345bd/users/c17a6b099ec14068b1e465024f75ed1d/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:10.914517 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24632b4f-ab74-4b39-a5c0-41ac3266ada8 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:10.914804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24632b4f-ab74-4b39-a5c0-41ac3266ada8 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:10.915102 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24632b4f-ab74-4b39-a5c0-41ac3266ada8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:10.943452 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-24632b4f-ab74-4b39-a5c0-41ac3266ada8 None None] MFA Rules not processed for user `c17a6b099ec14068b1e465024f75ed1d`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:10.944919 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-24632b4f-ab74-4b39-a5c0-41ac3266ada8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:11.046662 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1076/2175] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:11.051498 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a7a8d-ae4e-4c39-a3de-16c5a7f3e9a2 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:11.128291 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a7a8d-ae4e-4c39-a3de-16c5a7f3e9a2 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:11.133698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca132188-1e4b-4090-ac10-ba36136a097a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:11.134481 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca132188-1e4b-4090-ac10-ba36136a097a admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:11.134875 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca132188-1e4b-4090-ac10-ba36136a097a admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:11.135184 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca132188-1e4b-4090-ac10-ba36136a097a admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:11.135941 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca132188-1e4b-4090-ac10-ba36136a097a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:11.137930 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca132188-1e4b-4090-ac10-ba36136a097a admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:11.170751 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1100/2176] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:58:11 2022] POST /identity/v3/projects => generated 381 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:11.174854 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24632b4f-ab74-4b39-a5c0-41ac3266ada8 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:11.186673 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24632b4f-ab74-4b39-a5c0-41ac3266ada8 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:11.245997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e5ff9-30a3-4135-98db-2fde489322f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:11.246818 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e5ff9-30a3-4135-98db-2fde489322f1 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:11.247198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e5ff9-30a3-4135-98db-2fde489322f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:11.247467 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e5ff9-30a3-4135-98db-2fde489322f1 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:11.248181 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e5ff9-30a3-4135-98db-2fde489322f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:11.250173 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e5ff9-30a3-4135-98db-2fde489322f1 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:11.270697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1077/2177] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:58:11 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:11.274368 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca132188-1e4b-4090-ac10-ba36136a097a admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:11.284904 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca132188-1e4b-4090-ac10-ba36136a097a admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:11.290140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215bc5f8-ecaf-496e-af0c-3231c159a9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:11.290808 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215bc5f8-ecaf-496e-af0c-3231c159a9f9 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:11.291089 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215bc5f8-ecaf-496e-af0c-3231c159a9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:11.291363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215bc5f8-ecaf-496e-af0c-3231c159a9f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:11.292081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215bc5f8-ecaf-496e-af0c-3231c159a9f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:11.293795 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215bc5f8-ecaf-496e-af0c-3231c159a9f9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:11.313553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1101/2178] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:58:11.318414 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e5ff9-30a3-4135-98db-2fde489322f1 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:11.329340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e5ff9-30a3-4135-98db-2fde489322f1 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:11.334029 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621b70e-ad10-4521-9f64-34e558fb219b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:11.334984 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621b70e-ad10-4521-9f64-34e558fb219b admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:11.335266 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621b70e-ad10-4521-9f64-34e558fb219b admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:11.335539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621b70e-ad10-4521-9f64-34e558fb219b admin admin] PATH_INFO: `/v3/projects/6c05b736a92f4c6b9bfed1d945484a6a/users/0c9227e51f4a43478c8ebd9b5bd123f1/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:11.351281 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621b70e-ad10-4521-9f64-34e558fb219b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c05b736a92f4c6b9bfed1d945484a6a, user_id=0c9227e51f4a43478c8ebd9b5bd123f1, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:11.354505 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621b70e-ad10-4521-9f64-34e558fb219b admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:11.377537 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1078/2179] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:58:11 2022] PUT /identity/v3/projects/6c05b736a92f4c6b9bfed1d945484a6a/users/0c9227e51f4a43478c8ebd9b5bd123f1/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:58:11.382581 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215bc5f8-ecaf-496e-af0c-3231c159a9f9 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:11.393037 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215bc5f8-ecaf-496e-af0c-3231c159a9f9 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:11.475028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00557fbd-291e-47e9-acd3-340f41c6f0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:11.476037 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00557fbd-291e-47e9-acd3-340f41c6f0d6 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:11.476532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00557fbd-291e-47e9-acd3-340f41c6f0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:11.477208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00557fbd-291e-47e9-acd3-340f41c6f0d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:11.478872 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00557fbd-291e-47e9-acd3-340f41c6f0d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:11.480727 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00557fbd-291e-47e9-acd3-340f41c6f0d6 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:11.483787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:11.494526 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1102/2180] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:11.499214 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621b70e-ad10-4521-9f64-34e558fb219b admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:11.510460 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621b70e-ad10-4521-9f64-34e558fb219b admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:11.543512 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83434a68-c6d0-4ea6-8739-80114290aaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:11.544444 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83434a68-c6d0-4ea6-8739-80114290aaef admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:11.544716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83434a68-c6d0-4ea6-8739-80114290aaef admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:11.544988 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83434a68-c6d0-4ea6-8739-80114290aaef admin admin] PATH_INFO: `/v3/projects/6c05b736a92f4c6b9bfed1d945484a6a/users/0c9227e51f4a43478c8ebd9b5bd123f1/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:11.547030 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83434a68-c6d0-4ea6-8739-80114290aaef admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c05b736a92f4c6b9bfed1d945484a6a, user_id=0c9227e51f4a43478c8ebd9b5bd123f1, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:11.549686 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83434a68-c6d0-4ea6-8739-80114290aaef admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:11.552174 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:11.561774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1079/2181] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:58:11 2022] PUT /identity/v3/projects/6c05b736a92f4c6b9bfed1d945484a6a/users/0c9227e51f4a43478c8ebd9b5bd123f1/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:11.573174 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a317f2b-944d-423e-bb33-cdeb967971a3 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:11.573404 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a317f2b-944d-423e-bb33-cdeb967971a3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:11.573688 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a317f2b-944d-423e-bb33-cdeb967971a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:11.605390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-9a317f2b-944d-423e-bb33-cdeb967971a3 None None] MFA Rules not processed for user `0c9227e51f4a43478c8ebd9b5bd123f1`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:11.606640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9a317f2b-944d-423e-bb33-cdeb967971a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:11.698833 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1103/2182] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:11 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:11.710099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83434a68-c6d0-4ea6-8739-80114290aaef admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:11.712443 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-83434a68-c6d0-4ea6-8739-80114290aaef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:11.772099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83434a68-c6d0-4ea6-8739-80114290aaef admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:11.776373 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539d56b-b4f4-487f-ae1c-56072aa00137 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:11.777007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539d56b-b4f4-487f-ae1c-56072aa00137 service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:11.777286 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539d56b-b4f4-487f-ae1c-56072aa00137 service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:11.777559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539d56b-b4f4-487f-ae1c-56072aa00137 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:11.786617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539d56b-b4f4-487f-ae1c-56072aa00137 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:11.788060 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539d56b-b4f4-487f-ae1c-56072aa00137 service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:11.799283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1080/2183] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:12.302377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a317f2b-944d-423e-bb33-cdeb967971a3 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:12.370290 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a317f2b-944d-423e-bb33-cdeb967971a3 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:12.374844 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ba69c-8293-4291-9e6d-331915f92c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:12.375717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ba69c-8293-4291-9e6d-331915f92c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:12.376003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ba69c-8293-4291-9e6d-331915f92c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:12.376277 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ba69c-8293-4291-9e6d-331915f92c69 admin admin] PATH_INFO: `/v3/users/c17a6b099ec14068b1e465024f75ed1d` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:12.377463 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ba69c-8293-4291-9e6d-331915f92c69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c17a6b099ec14068b1e465024f75ed1d)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:12.379448 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ba69c-8293-4291-9e6d-331915f92c69 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:12.428827 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2f0ba69c-8293-4291-9e6d-331915f92c69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c17a6b099ec14068b1e465024f75ed1d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:12.442731 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2f0ba69c-8293-4291-9e6d-331915f92c69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c17a6b099ec14068b1e465024f75ed1d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:12.452911 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2f0ba69c-8293-4291-9e6d-331915f92c69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c17a6b099ec14068b1e465024f75ed1d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:12.453652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2f0ba69c-8293-4291-9e6d-331915f92c69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c17a6b099ec14068b1e465024f75ed1d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:12.454326 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2f0ba69c-8293-4291-9e6d-331915f92c69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c17a6b099ec14068b1e465024f75ed1d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:12.466894 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1104/2184] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:58:12 2022] DELETE /identity/v3/users/c17a6b099ec14068b1e465024f75ed1d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:12.472100 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad28b83-bb7b-4ce7-ab8e-c37984d87a5d None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:12.472407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad28b83-bb7b-4ce7-ab8e-c37984d87a5d None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:12.472830 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad28b83-bb7b-4ce7-ab8e-c37984d87a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:12.491878 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-bad28b83-bb7b-4ce7-ab8e-c37984d87a5d None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:12.493869 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-bad28b83-bb7b-4ce7-ab8e-c37984d87a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:12.558329 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1081/2185] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:58:12.569871 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ba69c-8293-4291-9e6d-331915f92c69 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:12.571944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2f0ba69c-8293-4291-9e6d-331915f92c69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:12.582126 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:12.659619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ba69c-8293-4291-9e6d-331915f92c69 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:12.663837 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e4fc4-fb38-4960-9938-37a21c3adc11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:12.670987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e4fc4-fb38-4960-9938-37a21c3adc11 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:12.671326 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e4fc4-fb38-4960-9938-37a21c3adc11 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:12.671690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e4fc4-fb38-4960-9938-37a21c3adc11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:12.685176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e4fc4-fb38-4960-9938-37a21c3adc11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:12.687544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e4fc4-fb38-4960-9938-37a21c3adc11 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:12.698670 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1105/2186] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:12.995599 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad28b83-bb7b-4ce7-ab8e-c37984d87a5d None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:12.997184 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-bad28b83-bb7b-4ce7-ab8e-c37984d87a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:13.012038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad28b83-bb7b-4ce7-ab8e-c37984d87a5d None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:13.017692 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283cd75-3b8a-4b7b-a2ab-a36df5f028c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:13.018842 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283cd75-3b8a-4b7b-a2ab-a36df5f028c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:13.019256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283cd75-3b8a-4b7b-a2ab-a36df5f028c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:13.019851 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283cd75-3b8a-4b7b-a2ab-a36df5f028c7 admin admin] PATH_INFO: `/v3/projects/b93a528b4b014972acd9e92c28e345bd` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:13.021457 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1283cd75-3b8a-4b7b-a2ab-a36df5f028c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b93a528b4b014972acd9e92c28e345bd)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:13.025370 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1283cd75-3b8a-4b7b-a2ab-a36df5f028c7 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:13.081537 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-1283cd75-3b8a-4b7b-a2ab-a36df5f028c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b93a528b4b014972acd9e92c28e345bd', 'request_id': 'req-1283cd75-3b8a-4b7b-a2ab-a36df5f028c7'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:13.088605 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-1283cd75-3b8a-4b7b-a2ab-a36df5f028c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b93a528b4b014972acd9e92c28e345bd', 'request_id': 'req-1283cd75-3b8a-4b7b-a2ab-a36df5f028c7'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:13.095785 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-1283cd75-3b8a-4b7b-a2ab-a36df5f028c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b93a528b4b014972acd9e92c28e345bd', 'request_id': 'req-1283cd75-3b8a-4b7b-a2ab-a36df5f028c7'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:13.115097 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-1283cd75-3b8a-4b7b-a2ab-a36df5f028c7 admin admin] The token cache is being invalidate because project b93a528b4b014972acd9e92c28e345bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:13.115393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-1283cd75-3b8a-4b7b-a2ab-a36df5f028c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:13.116183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-1283cd75-3b8a-4b7b-a2ab-a36df5f028c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:13.117966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1082/2187] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:58:12 2022] DELETE /identity/v3/projects/b93a528b4b014972acd9e92c28e345bd => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:13.121885 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e4fc4-fb38-4960-9938-37a21c3adc11 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:13.134556 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e4fc4-fb38-4960-9938-37a21c3adc11 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:13.209557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee944049-8342-40b6-a20c-27cc3825b311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:13.210391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee944049-8342-40b6-a20c-27cc3825b311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:13.210728 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee944049-8342-40b6-a20c-27cc3825b311 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:13.210949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee944049-8342-40b6-a20c-27cc3825b311 admin admin] PATH_INFO: `/v3/users/0c9227e51f4a43478c8ebd9b5bd123f1` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:13.212746 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee944049-8342-40b6-a20c-27cc3825b311 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c9227e51f4a43478c8ebd9b5bd123f1)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:13.214763 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee944049-8342-40b6-a20c-27cc3825b311 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:13.274504 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-ee944049-8342-40b6-a20c-27cc3825b311 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c9227e51f4a43478c8ebd9b5bd123f1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:13.287753 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-ee944049-8342-40b6-a20c-27cc3825b311 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c9227e51f4a43478c8ebd9b5bd123f1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:13.296187 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-ee944049-8342-40b6-a20c-27cc3825b311 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c9227e51f4a43478c8ebd9b5bd123f1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:13.296832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-ee944049-8342-40b6-a20c-27cc3825b311 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c9227e51f4a43478c8ebd9b5bd123f1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:13.297452 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-ee944049-8342-40b6-a20c-27cc3825b311 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c9227e51f4a43478c8ebd9b5bd123f1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:13.311928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1106/2188] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:58:13 2022] DELETE /identity/v3/users/0c9227e51f4a43478c8ebd9b5bd123f1 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:13.635966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283cd75-3b8a-4b7b-a2ab-a36df5f028c7 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:13.637603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-1283cd75-3b8a-4b7b-a2ab-a36df5f028c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:13.651516 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283cd75-3b8a-4b7b-a2ab-a36df5f028c7 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:13.671397 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:13.735138 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a7293-f909-4059-a00b-ad05a0dc9769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:13.737525 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a7293-f909-4059-a00b-ad05a0dc9769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:13.737525 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a7293-f909-4059-a00b-ad05a0dc9769 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:13.737525 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a7293-f909-4059-a00b-ad05a0dc9769 admin admin] PATH_INFO: `/v3/projects/6c05b736a92f4c6b9bfed1d945484a6a` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:13.738600 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a7293-f909-4059-a00b-ad05a0dc9769 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c05b736a92f4c6b9bfed1d945484a6a)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:13.740537 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a7293-f909-4059-a00b-ad05a0dc9769 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:13.802518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2d5a7293-f909-4059-a00b-ad05a0dc9769 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c05b736a92f4c6b9bfed1d945484a6a', 'request_id': 'req-2d5a7293-f909-4059-a00b-ad05a0dc9769'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:13.812404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2d5a7293-f909-4059-a00b-ad05a0dc9769 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c05b736a92f4c6b9bfed1d945484a6a', 'request_id': 'req-2d5a7293-f909-4059-a00b-ad05a0dc9769'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:13.827875 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2d5a7293-f909-4059-a00b-ad05a0dc9769 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c05b736a92f4c6b9bfed1d945484a6a', 'request_id': 'req-2d5a7293-f909-4059-a00b-ad05a0dc9769'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:13.838612 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2d5a7293-f909-4059-a00b-ad05a0dc9769 admin admin] The token cache is being invalidate because project 6c05b736a92f4c6b9bfed1d945484a6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:13.838997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2d5a7293-f909-4059-a00b-ad05a0dc9769 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:13.839757 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2d5a7293-f909-4059-a00b-ad05a0dc9769 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:13.841731 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1083/2189] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:58:13 2022] DELETE /identity/v3/projects/6c05b736a92f4c6b9bfed1d945484a6a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:13.848611 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61640065-787d-4473-ab26-a3488ce44b97 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:13.848979 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61640065-787d-4473-ab26-a3488ce44b97 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:13.849339 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61640065-787d-4473-ab26-a3488ce44b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:13.870338 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-61640065-787d-4473-ab26-a3488ce44b97 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:13.871644 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-61640065-787d-4473-ab26-a3488ce44b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:13.892937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:13.955113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1107/2190] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:58:13.963805 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95134b0-d73a-4200-b0e6-d6e00930fad5 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:13.964244 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95134b0-d73a-4200-b0e6-d6e00930fad5 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:13.964683 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95134b0-d73a-4200-b0e6-d6e00930fad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:13.984189 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-c95134b0-d73a-4200-b0e6-d6e00930fad5 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:13.985420 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-c95134b0-d73a-4200-b0e6-d6e00930fad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:13.995224 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1084/2191] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:14.000371 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61640065-787d-4473-ab26-a3488ce44b97 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:14.015660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61640065-787d-4473-ab26-a3488ce44b97 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:14.022682 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19504fd-4c27-4f8f-9710-563c7aadba03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:14.024290 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19504fd-4c27-4f8f-9710-563c7aadba03 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:14.024290 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19504fd-4c27-4f8f-9710-563c7aadba03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:14.024728 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19504fd-4c27-4f8f-9710-563c7aadba03 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:14.025673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19504fd-4c27-4f8f-9710-563c7aadba03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:58:14.025987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19504fd-4c27-4f8f-9710-563c7aadba03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:14.027917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19504fd-4c27-4f8f-9710-563c7aadba03 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:14.042109 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1108/2192] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:58:14.047117 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95134b0-d73a-4200-b0e6-d6e00930fad5 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:14.056590 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95134b0-d73a-4200-b0e6-d6e00930fad5 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:14.061791 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcafbef-01b0-42e4-9042-e81b5b5861bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:14.062473 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcafbef-01b0-42e4-9042-e81b5b5861bf admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:14.063252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcafbef-01b0-42e4-9042-e81b5b5861bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:14.063529 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcafbef-01b0-42e4-9042-e81b5b5861bf admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:14.064231 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcafbef-01b0-42e4-9042-e81b5b5861bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:14.066168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcafbef-01b0-42e4-9042-e81b5b5861bf admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:14.093143 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1085/2193] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:58:14 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:58:14.096655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19504fd-4c27-4f8f-9710-563c7aadba03 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:14.107252 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19504fd-4c27-4f8f-9710-563c7aadba03 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:14.165386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b703f8-0e42-4f05-9958-99a3964b0b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:14.166063 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b703f8-0e42-4f05-9958-99a3964b0b93 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:14.166353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b703f8-0e42-4f05-9958-99a3964b0b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:14.166632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b703f8-0e42-4f05-9958-99a3964b0b93 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:14.167332 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b703f8-0e42-4f05-9958-99a3964b0b93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:14.169265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b703f8-0e42-4f05-9958-99a3964b0b93 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:14.191566 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1109/2194] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:58:14 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:14.195854 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcafbef-01b0-42e4-9042-e81b5b5861bf admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:14.206331 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcafbef-01b0-42e4-9042-e81b5b5861bf admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:14.211050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892130fb-3d67-4f9b-9525-13e2551e3c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:14.211706 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892130fb-3d67-4f9b-9525-13e2551e3c3a admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:14.211983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892130fb-3d67-4f9b-9525-13e2551e3c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:14.212257 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892130fb-3d67-4f9b-9525-13e2551e3c3a admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:14.212991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892130fb-3d67-4f9b-9525-13e2551e3c3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:14.214611 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892130fb-3d67-4f9b-9525-13e2551e3c3a admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:14.229474 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1086/2195] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:58:14.233842 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b703f8-0e42-4f05-9958-99a3964b0b93 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:14.245851 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b703f8-0e42-4f05-9958-99a3964b0b93 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:14.250465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f69fc27-671a-4b2f-878e-0e1e7f4a331e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:14.251176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69fc27-671a-4b2f-878e-0e1e7f4a331e admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:14.251459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69fc27-671a-4b2f-878e-0e1e7f4a331e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:14.251736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69fc27-671a-4b2f-878e-0e1e7f4a331e admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:14.252462 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f69fc27-671a-4b2f-878e-0e1e7f4a331e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:14.254233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f69fc27-671a-4b2f-878e-0e1e7f4a331e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:14.266524 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1110/2196] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:14.271075 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892130fb-3d67-4f9b-9525-13e2551e3c3a admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:14.282698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892130fb-3d67-4f9b-9525-13e2551e3c3a admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:14.288385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93cd85b-4f50-4f0d-a6f2-194bb24f8748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:14.289957 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93cd85b-4f50-4f0d-a6f2-194bb24f8748 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:14.290165 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93cd85b-4f50-4f0d-a6f2-194bb24f8748 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:14.290502 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93cd85b-4f50-4f0d-a6f2-194bb24f8748 admin admin] PATH_INFO: `/v3/projects/147f6f739c6e4e1bab0b20768f3131c2/users/26a977b132994a42a0c830224ca1c1f8/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:14.307714 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93cd85b-4f50-4f0d-a6f2-194bb24f8748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=147f6f739c6e4e1bab0b20768f3131c2, user_id=26a977b132994a42a0c830224ca1c1f8, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:14.310483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93cd85b-4f50-4f0d-a6f2-194bb24f8748 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:14.323559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1087/2197] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:58:14 2022] PUT /identity/v3/projects/147f6f739c6e4e1bab0b20768f3131c2/users/26a977b132994a42a0c830224ca1c1f8/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:14.334001 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4583ee3-28f3-4ae7-b902-25c754ec979b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:14.334605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4583ee3-28f3-4ae7-b902-25c754ec979b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:14.334705 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4583ee3-28f3-4ae7-b902-25c754ec979b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:14.365313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b4583ee3-28f3-4ae7-b902-25c754ec979b None None] MFA Rules not processed for user `26a977b132994a42a0c830224ca1c1f8`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:14.367284 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b4583ee3-28f3-4ae7-b902-25c754ec979b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:14.438426 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1111/2198] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:14.450408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93cd85b-4f50-4f0d-a6f2-194bb24f8748 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:14.452122 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d93cd85b-4f50-4f0d-a6f2-194bb24f8748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:14.544359 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:14.576744 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93cd85b-4f50-4f0d-a6f2-194bb24f8748 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:14.580665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe667589-1d6e-440f-b4b3-2ba4170bbed2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88cbd76f70d447c4ba92777b2c09821d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:14.581297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe667589-1d6e-440f-b4b3-2ba4170bbed2 service glance] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:14.581577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe667589-1d6e-440f-b4b3-2ba4170bbed2 service glance] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:14.581851 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe667589-1d6e-440f-b4b3-2ba4170bbed2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:14.590813 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe667589-1d6e-440f-b4b3-2ba4170bbed2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:14.592353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe667589-1d6e-440f-b4b3-2ba4170bbed2 service glance] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:14.602270 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1088/2199] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:14.769807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4583ee3-28f3-4ae7-b902-25c754ec979b None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:14.771646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b4583ee3-28f3-4ae7-b902-25c754ec979b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:14.799089 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:14.846860 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4583ee3-28f3-4ae7-b902-25c754ec979b None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:14.850385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16e44d-24af-4364-9ea0-c16c9c949bd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:14.851256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16e44d-24af-4364-9ea0-c16c9c949bd8 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:14.851497 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16e44d-24af-4364-9ea0-c16c9c949bd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:14.851589 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16e44d-24af-4364-9ea0-c16c9c949bd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:14.854312 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2b16e44d-24af-4364-9ea0-c16c9c949bd8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:14.941084 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16e44d-24af-4364-9ea0-c16c9c949bd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:14.942621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16e44d-24af-4364-9ea0-c16c9c949bd8 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:14.954061 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1112/2200] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:14 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:24.597566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=14 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:24.949151 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=41 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:44.151276 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe667589-1d6e-440f-b4b3-2ba4170bbed2 service glance] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:44.153016 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-fe667589-1d6e-440f-b4b3-2ba4170bbed2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:44.249732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe667589-1d6e-440f-b4b3-2ba4170bbed2 service glance] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:44.257838 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fbf43-681e-4270-81b6-6b2d520d7d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:44.259405 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fbf43-681e-4270-81b6-6b2d520d7d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:44.259903 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fbf43-681e-4270-81b6-6b2d520d7d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:44.260410 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fbf43-681e-4270-81b6-6b2d520d7d1b admin admin] PATH_INFO: `/v3/users/cd9c8b056d80474b95b14f04ccc6c407` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:44.262623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026fbf43-681e-4270-81b6-6b2d520d7d1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd9c8b056d80474b95b14f04ccc6c407)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:44.264741 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026fbf43-681e-4270-81b6-6b2d520d7d1b admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:44.351328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-026fbf43-681e-4270-81b6-6b2d520d7d1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd9c8b056d80474b95b14f04ccc6c407'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:44.387901 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-026fbf43-681e-4270-81b6-6b2d520d7d1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd9c8b056d80474b95b14f04ccc6c407'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:44.403637 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-026fbf43-681e-4270-81b6-6b2d520d7d1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd9c8b056d80474b95b14f04ccc6c407'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:44.404675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-026fbf43-681e-4270-81b6-6b2d520d7d1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd9c8b056d80474b95b14f04ccc6c407'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:44.406489 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-026fbf43-681e-4270-81b6-6b2d520d7d1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd9c8b056d80474b95b14f04ccc6c407'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:44.433650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1089/2201] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:58:44 2022] DELETE /identity/v3/users/cd9c8b056d80474b95b14f04ccc6c407 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:44.441819 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f46d5-6708-4eb3-bc37-0dcbd5d0519a None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:44.442461 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f46d5-6708-4eb3-bc37-0dcbd5d0519a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:44.443578 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f46d5-6708-4eb3-bc37-0dcbd5d0519a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:44.472133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-3b9f46d5-6708-4eb3-bc37-0dcbd5d0519a None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:44.473724 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3b9f46d5-6708-4eb3-bc37-0dcbd5d0519a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:44.568351 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1113/2202] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:44.579233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fbf43-681e-4270-81b6-6b2d520d7d1b admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:44.580542 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-026fbf43-681e-4270-81b6-6b2d520d7d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:44.659140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fbf43-681e-4270-81b6-6b2d520d7d1b admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:44.662796 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84db349-736d-4975-b48e-90afa2e5c00d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:44.663448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84db349-736d-4975-b48e-90afa2e5c00d service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:44.663730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84db349-736d-4975-b48e-90afa2e5c00d service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:44.664002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84db349-736d-4975-b48e-90afa2e5c00d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:44.674468 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84db349-736d-4975-b48e-90afa2e5c00d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:44.675923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84db349-736d-4975-b48e-90afa2e5c00d service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:44.686909 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1090/2203] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:45.010917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f46d5-6708-4eb3-bc37-0dcbd5d0519a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:45.012803 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3b9f46d5-6708-4eb3-bc37-0dcbd5d0519a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:45.044992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f46d5-6708-4eb3-bc37-0dcbd5d0519a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:45.058680 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793cb515-0c1c-45e7-b87f-babd30efe34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:45.061008 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793cb515-0c1c-45e7-b87f-babd30efe34e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:45.061409 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793cb515-0c1c-45e7-b87f-babd30efe34e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:45.061964 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793cb515-0c1c-45e7-b87f-babd30efe34e admin admin] PATH_INFO: `/v3/projects/30e183b982804c7b858ff95dd533decf` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:45.064137 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793cb515-0c1c-45e7-b87f-babd30efe34e admin admin] RBAC: Authorizing `identity:delete_project(project_id=30e183b982804c7b858ff95dd533decf)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:45.066986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793cb515-0c1c-45e7-b87f-babd30efe34e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:45.148286 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-793cb515-0c1c-45e7-b87f-babd30efe34e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30e183b982804c7b858ff95dd533decf', 'request_id': 'req-793cb515-0c1c-45e7-b87f-babd30efe34e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:45.158177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-793cb515-0c1c-45e7-b87f-babd30efe34e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30e183b982804c7b858ff95dd533decf', 'request_id': 'req-793cb515-0c1c-45e7-b87f-babd30efe34e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:45.169090 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-793cb515-0c1c-45e7-b87f-babd30efe34e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30e183b982804c7b858ff95dd533decf', 'request_id': 'req-793cb515-0c1c-45e7-b87f-babd30efe34e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:45.190003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-793cb515-0c1c-45e7-b87f-babd30efe34e admin admin] The token cache is being invalidate because project 30e183b982804c7b858ff95dd533decf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:45.190471 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-793cb515-0c1c-45e7-b87f-babd30efe34e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:45.191491 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-793cb515-0c1c-45e7-b87f-babd30efe34e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:45.194114 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1114/2204] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:58:45 2022] DELETE /identity/v3/projects/30e183b982804c7b858ff95dd533decf => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:45.201514 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a920e8a-0d9e-428c-be9b-4fa9a0b016c8 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:45.202347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a920e8a-0d9e-428c-be9b-4fa9a0b016c8 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:45.202800 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a920e8a-0d9e-428c-be9b-4fa9a0b016c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:45.223371 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-0a920e8a-0d9e-428c-be9b-4fa9a0b016c8 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:45.224596 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-0a920e8a-0d9e-428c-be9b-4fa9a0b016c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:45.254703 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:45.324270 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1091/2205] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:58:45.332510 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57b8fc-090f-450b-bb1e-d4a969945c62 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:45.332808 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57b8fc-090f-450b-bb1e-d4a969945c62 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:45.333132 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57b8fc-090f-450b-bb1e-d4a969945c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:45.371547 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-eb57b8fc-090f-450b-bb1e-d4a969945c62 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:45.373517 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-eb57b8fc-090f-450b-bb1e-d4a969945c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:45.392821 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1115/2206] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:45.397583 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a920e8a-0d9e-428c-be9b-4fa9a0b016c8 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:45.415743 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a920e8a-0d9e-428c-be9b-4fa9a0b016c8 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:45.421509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c87fb8-b74d-4361-8074-aaea8cec7455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:45.422590 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c87fb8-b74d-4361-8074-aaea8cec7455 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:45.423104 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c87fb8-b74d-4361-8074-aaea8cec7455 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:45.423475 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c87fb8-b74d-4361-8074-aaea8cec7455 admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:45.424932 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c87fb8-b74d-4361-8074-aaea8cec7455 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:58:45.425518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c87fb8-b74d-4361-8074-aaea8cec7455 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:45.427236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c87fb8-b74d-4361-8074-aaea8cec7455 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:45.445325 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1092/2207] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:58:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:58:45.450323 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57b8fc-090f-450b-bb1e-d4a969945c62 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:45.460331 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57b8fc-090f-450b-bb1e-d4a969945c62 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:45.468306 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef322b-9c6e-4367-a352-e2b00824814d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:45.469149 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef322b-9c6e-4367-a352-e2b00824814d admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:45.469595 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef322b-9c6e-4367-a352-e2b00824814d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:45.469851 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef322b-9c6e-4367-a352-e2b00824814d admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:45.470788 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ef322b-9c6e-4367-a352-e2b00824814d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:45.472847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ef322b-9c6e-4367-a352-e2b00824814d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:45.516058 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1116/2208] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:58:45 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:45.516775 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c87fb8-b74d-4361-8074-aaea8cec7455 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:45.535605 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c87fb8-b74d-4361-8074-aaea8cec7455 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:45.608561 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd6d56-e4ff-4b3f-999e-835f842b0220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:45.609893 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd6d56-e4ff-4b3f-999e-835f842b0220 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:45.610488 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd6d56-e4ff-4b3f-999e-835f842b0220 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:45.610957 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd6d56-e4ff-4b3f-999e-835f842b0220 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:45.612014 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd6d56-e4ff-4b3f-999e-835f842b0220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:45.614155 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd6d56-e4ff-4b3f-999e-835f842b0220 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:45.638047 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1093/2209] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:58:45 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:45.641956 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef322b-9c6e-4367-a352-e2b00824814d admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:45.653415 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef322b-9c6e-4367-a352-e2b00824814d admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:45.658634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ba2d2-9ee0-49f4-b2fe-43f7e7ec7583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:45.659388 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ba2d2-9ee0-49f4-b2fe-43f7e7ec7583 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:45.659770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ba2d2-9ee0-49f4-b2fe-43f7e7ec7583 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:45.660286 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ba2d2-9ee0-49f4-b2fe-43f7e7ec7583 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:45.661089 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ba2d2-9ee0-49f4-b2fe-43f7e7ec7583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:45.662811 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ba2d2-9ee0-49f4-b2fe-43f7e7ec7583 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:45.677574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1117/2210] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:45.682528 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd6d56-e4ff-4b3f-999e-835f842b0220 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:45.695762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd6d56-e4ff-4b3f-999e-835f842b0220 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:45.710368 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62bd41-da9a-4944-8f2d-a45da555c654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:45.710368 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62bd41-da9a-4944-8f2d-a45da555c654 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:45.710368 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62bd41-da9a-4944-8f2d-a45da555c654 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:45.710368 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62bd41-da9a-4944-8f2d-a45da555c654 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:45.710743 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62bd41-da9a-4944-8f2d-a45da555c654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:45.710743 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62bd41-da9a-4944-8f2d-a45da555c654 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:45.717557 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1094/2211] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:45.721743 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ba2d2-9ee0-49f4-b2fe-43f7e7ec7583 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:45.732775 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ba2d2-9ee0-49f4-b2fe-43f7e7ec7583 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:45.737709 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e559954-a020-4cf6-b04f-ed62efbca88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:45.738695 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e559954-a020-4cf6-b04f-ed62efbca88c admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:45.738974 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e559954-a020-4cf6-b04f-ed62efbca88c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:45.739250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e559954-a020-4cf6-b04f-ed62efbca88c admin admin] PATH_INFO: `/v3/projects/5d867311a11b4be482ffdfed4731a86f/users/6afcf92d1eeb476ba0a9b4cd38e68e02/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:45.757573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e559954-a020-4cf6-b04f-ed62efbca88c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d867311a11b4be482ffdfed4731a86f, user_id=6afcf92d1eeb476ba0a9b4cd38e68e02, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:45.760895 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e559954-a020-4cf6-b04f-ed62efbca88c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:45.776281 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1118/2212] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:58:45 2022] PUT /identity/v3/projects/5d867311a11b4be482ffdfed4731a86f/users/6afcf92d1eeb476ba0a9b4cd38e68e02/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:45.787458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d80c6b-657b-4537-af98-9b2039262c89 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:45.787768 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d80c6b-657b-4537-af98-9b2039262c89 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:45.788064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d80c6b-657b-4537-af98-9b2039262c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:45.816569 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-a8d80c6b-657b-4537-af98-9b2039262c89 None None] MFA Rules not processed for user `6afcf92d1eeb476ba0a9b4cd38e68e02`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:45.817787 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-a8d80c6b-657b-4537-af98-9b2039262c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:45.891569 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1095/2213] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:45.896478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e559954-a020-4cf6-b04f-ed62efbca88c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:45.921595 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:45.968263 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e559954-a020-4cf6-b04f-ed62efbca88c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:45.973053 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92346e5-4fff-4535-b635-0e3ff9445fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:45.973666 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92346e5-4fff-4535-b635-0e3ff9445fd1 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:45.973948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92346e5-4fff-4535-b635-0e3ff9445fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:45.974249 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92346e5-4fff-4535-b635-0e3ff9445fd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:45.974938 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92346e5-4fff-4535-b635-0e3ff9445fd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:45.976903 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92346e5-4fff-4535-b635-0e3ff9445fd1 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:46.008569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1119/2214] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:58:45 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:58:46.009711 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d80c6b-657b-4537-af98-9b2039262c89 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:46.020895 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d80c6b-657b-4537-af98-9b2039262c89 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:46.034278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:46.080757 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79242647-1dee-4535-8cea-57a624fdd405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:46.081436 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79242647-1dee-4535-8cea-57a624fdd405 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:46.081715 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79242647-1dee-4535-8cea-57a624fdd405 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:46.081991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79242647-1dee-4535-8cea-57a624fdd405 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:46.082715 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79242647-1dee-4535-8cea-57a624fdd405 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:46.084645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79242647-1dee-4535-8cea-57a624fdd405 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:46.106587 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1096/2215] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:58:46 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:58:46.110712 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92346e5-4fff-4535-b635-0e3ff9445fd1 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:46.122181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92346e5-4fff-4535-b635-0e3ff9445fd1 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:46.127541 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73b354c-1353-4eec-a49b-41b7a4f05de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:46.128217 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b354c-1353-4eec-a49b-41b7a4f05de6 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:46.128488 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b354c-1353-4eec-a49b-41b7a4f05de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:46.128763 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b354c-1353-4eec-a49b-41b7a4f05de6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:46.129528 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73b354c-1353-4eec-a49b-41b7a4f05de6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:46.131487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73b354c-1353-4eec-a49b-41b7a4f05de6 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:46.144073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1120/2216] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:46.148378 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79242647-1dee-4535-8cea-57a624fdd405 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:46.158826 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79242647-1dee-4535-8cea-57a624fdd405 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:46.163770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872d570-e5f7-4257-917e-4e0eb2285053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:46.164434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872d570-e5f7-4257-917e-4e0eb2285053 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:46.164707 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872d570-e5f7-4257-917e-4e0eb2285053 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:46.164978 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872d570-e5f7-4257-917e-4e0eb2285053 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:46.165707 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6872d570-e5f7-4257-917e-4e0eb2285053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:46.167342 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6872d570-e5f7-4257-917e-4e0eb2285053 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:46.177548 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1097/2217] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:46.182725 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73b354c-1353-4eec-a49b-41b7a4f05de6 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:46.192783 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73b354c-1353-4eec-a49b-41b7a4f05de6 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:46.197985 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d878ca-9f3b-41d5-81df-7f85d9396d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:46.198889 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d878ca-9f3b-41d5-81df-7f85d9396d27 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:46.199171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d878ca-9f3b-41d5-81df-7f85d9396d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:46.199443 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d878ca-9f3b-41d5-81df-7f85d9396d27 admin admin] PATH_INFO: `/v3/projects/cea6e5b216414e27890aca6d6176273d/users/64ebc54f27524a3ebc2f1e279c0fbe54/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:46.215750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d878ca-9f3b-41d5-81df-7f85d9396d27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cea6e5b216414e27890aca6d6176273d, user_id=64ebc54f27524a3ebc2f1e279c0fbe54, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:46.218538 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d878ca-9f3b-41d5-81df-7f85d9396d27 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:46.232086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1121/2218] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:58:46 2022] PUT /identity/v3/projects/cea6e5b216414e27890aca6d6176273d/users/64ebc54f27524a3ebc2f1e279c0fbe54/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:58:46.243921 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecceb243-525a-4754-97a6-261794920137 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:46.244438 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecceb243-525a-4754-97a6-261794920137 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:46.244529 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecceb243-525a-4754-97a6-261794920137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:46.272522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-ecceb243-525a-4754-97a6-261794920137 None None] MFA Rules not processed for user `64ebc54f27524a3ebc2f1e279c0fbe54`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:46.273739 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-ecceb243-525a-4754-97a6-261794920137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:46.340929 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1098/2219] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:46.345552 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d878ca-9f3b-41d5-81df-7f85d9396d27 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:46.412479 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d878ca-9f3b-41d5-81df-7f85d9396d27 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:46.418868 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da97109-5826-4f3d-b231-f0e76acb94d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:46.419810 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da97109-5826-4f3d-b231-f0e76acb94d5 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:46.420284 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da97109-5826-4f3d-b231-f0e76acb94d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:46.420565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da97109-5826-4f3d-b231-f0e76acb94d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:46.421395 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da97109-5826-4f3d-b231-f0e76acb94d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:46.423514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da97109-5826-4f3d-b231-f0e76acb94d5 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:46.452349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1122/2220] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:58:46 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:46.458600 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecceb243-525a-4754-97a6-261794920137 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:46.470641 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecceb243-525a-4754-97a6-261794920137 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:46.541774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f15bac-35ec-4fe0-94d5-a376594fcbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:46.542576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f15bac-35ec-4fe0-94d5-a376594fcbaa admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:46.542918 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f15bac-35ec-4fe0-94d5-a376594fcbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:46.543257 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f15bac-35ec-4fe0-94d5-a376594fcbaa admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:46.544093 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f15bac-35ec-4fe0-94d5-a376594fcbaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:46.546482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f15bac-35ec-4fe0-94d5-a376594fcbaa admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:46.580839 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1099/2221] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:58:46 2022] POST /identity/v3/users => generated 295 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:46.584655 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da97109-5826-4f3d-b231-f0e76acb94d5 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:46.600944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da97109-5826-4f3d-b231-f0e76acb94d5 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:46.606439 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfa174-22b4-4003-baa9-2c3c20897e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:46.607130 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfa174-22b4-4003-baa9-2c3c20897e64 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:46.607418 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfa174-22b4-4003-baa9-2c3c20897e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:46.607708 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfa174-22b4-4003-baa9-2c3c20897e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:46.612148 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cfa174-22b4-4003-baa9-2c3c20897e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:46.612148 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cfa174-22b4-4003-baa9-2c3c20897e64 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:46.627263 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1123/2222] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:58:46.631690 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f15bac-35ec-4fe0-94d5-a376594fcbaa admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:46.645796 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f15bac-35ec-4fe0-94d5-a376594fcbaa admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:46.652738 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a514c206-f4b8-4d59-a773-fd0ddeb650e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:46.653686 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a514c206-f4b8-4d59-a773-fd0ddeb650e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:46.653952 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a514c206-f4b8-4d59-a773-fd0ddeb650e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:46.654279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a514c206-f4b8-4d59-a773-fd0ddeb650e6 admin admin] PATH_INFO: `/v3/projects/0052632cfce148b3942fd1cd6cf92218/users/b0f4005528044659a8a24ab17b4937e1/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:46.673482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:46.678143 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a514c206-f4b8-4d59-a773-fd0ddeb650e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0052632cfce148b3942fd1cd6cf92218, user_id=b0f4005528044659a8a24ab17b4937e1, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:46.682376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a514c206-f4b8-4d59-a773-fd0ddeb650e6 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:46.705720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1100/2223] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:58:46 2022] PUT /identity/v3/projects/0052632cfce148b3942fd1cd6cf92218/users/b0f4005528044659a8a24ab17b4937e1/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:58:46.710402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfa174-22b4-4003-baa9-2c3c20897e64 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:46.724855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfa174-22b4-4003-baa9-2c3c20897e64 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:46.785343 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:46.812066 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff2d13-c7b5-47c6-b8d5-e4bc33b634be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:46.812745 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff2d13-c7b5-47c6-b8d5-e4bc33b634be admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:46.813033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff2d13-c7b5-47c6-b8d5-e4bc33b634be admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:46.813336 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff2d13-c7b5-47c6-b8d5-e4bc33b634be admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:46.814148 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff2d13-c7b5-47c6-b8d5-e4bc33b634be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:46.815797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff2d13-c7b5-47c6-b8d5-e4bc33b634be admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:46.827484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1124/2224] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:46.832025 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a514c206-f4b8-4d59-a773-fd0ddeb650e6 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:46.845407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a514c206-f4b8-4d59-a773-fd0ddeb650e6 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:46.850392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5e15c-0dbe-4f08-bdce-8279788abeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:46.851335 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5e15c-0dbe-4f08-bdce-8279788abeba admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:46.851632 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5e15c-0dbe-4f08-bdce-8279788abeba admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:46.851862 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5e15c-0dbe-4f08-bdce-8279788abeba admin admin] PATH_INFO: `/v3/projects/0052632cfce148b3942fd1cd6cf92218/users/b0f4005528044659a8a24ab17b4937e1/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:46.854317 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d5e15c-0dbe-4f08-bdce-8279788abeba admin admin] RBAC: Authorizing `identity:create_grant(project_id=0052632cfce148b3942fd1cd6cf92218, user_id=b0f4005528044659a8a24ab17b4937e1, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:46.857130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d5e15c-0dbe-4f08-bdce-8279788abeba admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:46.870299 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1101/2225] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:58:46 2022] PUT /identity/v3/projects/0052632cfce148b3942fd1cd6cf92218/users/b0f4005528044659a8a24ab17b4937e1/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:58:46.880922 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807500d-cab4-4b3b-90e7-42dc43a905bd None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:46.881216 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807500d-cab4-4b3b-90e7-42dc43a905bd None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:46.881496 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807500d-cab4-4b3b-90e7-42dc43a905bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:46.917396 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-5807500d-cab4-4b3b-90e7-42dc43a905bd None None] MFA Rules not processed for user `b0f4005528044659a8a24ab17b4937e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:46.918792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5807500d-cab4-4b3b-90e7-42dc43a905bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:46.991853 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1125/2226] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:47.003970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5e15c-0dbe-4f08-bdce-8279788abeba admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:47.005729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-54d5e15c-0dbe-4f08-bdce-8279788abeba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:47.065529 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5e15c-0dbe-4f08-bdce-8279788abeba admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:47.069284 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355ec0d-29ce-49e0-90f5-1087d3cdb11c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:47.069935 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355ec0d-29ce-49e0-90f5-1087d3cdb11c service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:47.070251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355ec0d-29ce-49e0-90f5-1087d3cdb11c service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:47.070527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355ec0d-29ce-49e0-90f5-1087d3cdb11c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:47.127878 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355ec0d-29ce-49e0-90f5-1087d3cdb11c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:47.129349 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355ec0d-29ce-49e0-90f5-1087d3cdb11c service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:47.139173 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1102/2227] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:47.735301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807500d-cab4-4b3b-90e7-42dc43a905bd None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:47.736853 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5807500d-cab4-4b3b-90e7-42dc43a905bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:47.828809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807500d-cab4-4b3b-90e7-42dc43a905bd None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:47.851377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9174df-1306-4f9d-83f0-693a6f5f0fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:47.852393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9174df-1306-4f9d-83f0-693a6f5f0fba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:47.852723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9174df-1306-4f9d-83f0-693a6f5f0fba admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:47.853008 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9174df-1306-4f9d-83f0-693a6f5f0fba admin admin] PATH_INFO: `/v3/users/0d313b61fa24407b846372b796701242` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:47.854686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9174df-1306-4f9d-83f0-693a6f5f0fba admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d313b61fa24407b846372b796701242)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:47.856775 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9174df-1306-4f9d-83f0-693a6f5f0fba admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:47.919659 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-da9174df-1306-4f9d-83f0-693a6f5f0fba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d313b61fa24407b846372b796701242'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:47.942696 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-da9174df-1306-4f9d-83f0-693a6f5f0fba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d313b61fa24407b846372b796701242'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:47.945412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:47.953950 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-da9174df-1306-4f9d-83f0-693a6f5f0fba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d313b61fa24407b846372b796701242'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:47.955047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-da9174df-1306-4f9d-83f0-693a6f5f0fba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d313b61fa24407b846372b796701242'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:47.955966 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-da9174df-1306-4f9d-83f0-693a6f5f0fba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d313b61fa24407b846372b796701242'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:47.974078 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1126/2228] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:58:47 2022] DELETE /identity/v3/users/0d313b61fa24407b846372b796701242 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:47.978532 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17c81c-687e-4803-b422-917bf537c01c None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:47.979009 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17c81c-687e-4803-b422-917bf537c01c None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:47.979260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17c81c-687e-4803-b422-917bf537c01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:48.000264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-0f17c81c-687e-4803-b422-917bf537c01c None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:48.001746 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-0f17c81c-687e-4803-b422-917bf537c01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:48.077738 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1103/2229] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:58:48.088204 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9174df-1306-4f9d-83f0-693a6f5f0fba admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:48.089747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-da9174df-1306-4f9d-83f0-693a6f5f0fba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:48.156729 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9174df-1306-4f9d-83f0-693a6f5f0fba admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:48.160380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c41456-e73b-43a7-a1fc-ed322552c6dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:48.161275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c41456-e73b-43a7-a1fc-ed322552c6dc service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:48.161491 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c41456-e73b-43a7-a1fc-ed322552c6dc service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:48.161583 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c41456-e73b-43a7-a1fc-ed322552c6dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:48.171046 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c41456-e73b-43a7-a1fc-ed322552c6dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:48.172493 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c41456-e73b-43a7-a1fc-ed322552c6dc service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:48.183915 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1127/2230] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:48.473839 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17c81c-687e-4803-b422-917bf537c01c None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:48.478218 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-0f17c81c-687e-4803-b422-917bf537c01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:48.501628 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17c81c-687e-4803-b422-917bf537c01c None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:48.506673 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03a006-1f49-454f-99fb-6c1ed997586d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:48.508182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03a006-1f49-454f-99fb-6c1ed997586d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:48.508462 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03a006-1f49-454f-99fb-6c1ed997586d admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:48.508733 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03a006-1f49-454f-99fb-6c1ed997586d admin admin] PATH_INFO: `/v3/projects/6e27eea48a204fb196e5bad0c061924b` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:48.510779 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03a006-1f49-454f-99fb-6c1ed997586d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e27eea48a204fb196e5bad0c061924b)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:48.513130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03a006-1f49-454f-99fb-6c1ed997586d admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:48.558898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:48.587182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-fb03a006-1f49-454f-99fb-6c1ed997586d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e27eea48a204fb196e5bad0c061924b', 'request_id': 'req-fb03a006-1f49-454f-99fb-6c1ed997586d'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:48.597118 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-fb03a006-1f49-454f-99fb-6c1ed997586d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e27eea48a204fb196e5bad0c061924b', 'request_id': 'req-fb03a006-1f49-454f-99fb-6c1ed997586d'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:48.605646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-fb03a006-1f49-454f-99fb-6c1ed997586d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e27eea48a204fb196e5bad0c061924b', 'request_id': 'req-fb03a006-1f49-454f-99fb-6c1ed997586d'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:48.623343 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-fb03a006-1f49-454f-99fb-6c1ed997586d admin admin] The token cache is being invalidate because project 6e27eea48a204fb196e5bad0c061924b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:48.623637 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-fb03a006-1f49-454f-99fb-6c1ed997586d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:48.624334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-fb03a006-1f49-454f-99fb-6c1ed997586d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:48.625977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1104/2231] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:58:48 2022] DELETE /identity/v3/projects/6e27eea48a204fb196e5bad0c061924b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:58:48.629934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c41456-e73b-43a7-a1fc-ed322552c6dc service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:48.643592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c41456-e73b-43a7-a1fc-ed322552c6dc service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:48.733776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03a006-1f49-454f-99fb-6c1ed997586d admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:48.738153 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-fb03a006-1f49-454f-99fb-6c1ed997586d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:48.751062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137facd-8203-4353-968c-bea5a7c9b44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:48.751980 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137facd-8203-4353-968c-bea5a7c9b44f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:48.752263 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137facd-8203-4353-968c-bea5a7c9b44f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:48.752540 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137facd-8203-4353-968c-bea5a7c9b44f admin admin] PATH_INFO: `/v3/users/61af1910640e45aeb66920c60e64cdbe` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:48.753798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137facd-8203-4353-968c-bea5a7c9b44f admin admin] RBAC: Authorizing `identity:delete_user(user_id=61af1910640e45aeb66920c60e64cdbe)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:48.756359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137facd-8203-4353-968c-bea5a7c9b44f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:48.817226 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03a006-1f49-454f-99fb-6c1ed997586d admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:48.821614 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06557f5-881d-4b9f-8b8b-0e614b6db850 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:48.822417 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06557f5-881d-4b9f-8b8b-0e614b6db850 service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:48.823904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06557f5-881d-4b9f-8b8b-0e614b6db850 service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:48.824298 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06557f5-881d-4b9f-8b8b-0e614b6db850 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:48.827736 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-c06557f5-881d-4b9f-8b8b-0e614b6db850 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:48.863316 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0137facd-8203-4353-968c-bea5a7c9b44f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61af1910640e45aeb66920c60e64cdbe'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:48.890619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0137facd-8203-4353-968c-bea5a7c9b44f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61af1910640e45aeb66920c60e64cdbe'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:48.913969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0137facd-8203-4353-968c-bea5a7c9b44f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61af1910640e45aeb66920c60e64cdbe'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:48.914726 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0137facd-8203-4353-968c-bea5a7c9b44f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61af1910640e45aeb66920c60e64cdbe'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:48.916020 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0137facd-8203-4353-968c-bea5a7c9b44f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61af1910640e45aeb66920c60e64cdbe'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:48.938896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1128/2232] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:58:48 2022] DELETE /identity/v3/users/61af1910640e45aeb66920c60e64cdbe => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:48.945853 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06557f5-881d-4b9f-8b8b-0e614b6db850 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:48.947430 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06557f5-881d-4b9f-8b8b-0e614b6db850 service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:48.958748 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1105/2233] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:58:49.357031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137facd-8203-4353-968c-bea5a7c9b44f admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:49.359380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-0137facd-8203-4353-968c-bea5a7c9b44f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:49.416120 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:49.521769 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137facd-8203-4353-968c-bea5a7c9b44f admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:49.529076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ba6a0-6aa6-4a9b-bdc4-59cdfa45cd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:49.530149 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ba6a0-6aa6-4a9b-bdc4-59cdfa45cd01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:49.530481 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ba6a0-6aa6-4a9b-bdc4-59cdfa45cd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:49.530836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ba6a0-6aa6-4a9b-bdc4-59cdfa45cd01 admin admin] PATH_INFO: `/v3/projects/72bd84196a454d5787a9a89726df6182` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:49.539333 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ba6a0-6aa6-4a9b-bdc4-59cdfa45cd01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72bd84196a454d5787a9a89726df6182)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:49.548943 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ba6a0-6aa6-4a9b-bdc4-59cdfa45cd01 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:49.647775 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4e9ba6a0-6aa6-4a9b-bdc4-59cdfa45cd01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72bd84196a454d5787a9a89726df6182', 'request_id': 'req-4e9ba6a0-6aa6-4a9b-bdc4-59cdfa45cd01'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:49.655687 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4e9ba6a0-6aa6-4a9b-bdc4-59cdfa45cd01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72bd84196a454d5787a9a89726df6182', 'request_id': 'req-4e9ba6a0-6aa6-4a9b-bdc4-59cdfa45cd01'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:49.663670 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4e9ba6a0-6aa6-4a9b-bdc4-59cdfa45cd01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72bd84196a454d5787a9a89726df6182', 'request_id': 'req-4e9ba6a0-6aa6-4a9b-bdc4-59cdfa45cd01'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:49.681566 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4e9ba6a0-6aa6-4a9b-bdc4-59cdfa45cd01 admin admin] The token cache is being invalidate because project 72bd84196a454d5787a9a89726df6182 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:49.681965 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4e9ba6a0-6aa6-4a9b-bdc4-59cdfa45cd01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:49.682902 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4e9ba6a0-6aa6-4a9b-bdc4-59cdfa45cd01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:49.688036 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1129/2234] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:58:49 2022] DELETE /identity/v3/projects/72bd84196a454d5787a9a89726df6182 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:49.691614 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f8e77-4da7-426d-b188-176b80d54f8e None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:49.692125 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f8e77-4da7-426d-b188-176b80d54f8e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:49.692125 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f8e77-4da7-426d-b188-176b80d54f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:49.706929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ba6a0-6aa6-4a9b-bdc4-59cdfa45cd01 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:49.710089 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4e9ba6a0-6aa6-4a9b-bdc4-59cdfa45cd01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:49.716778 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-185f8e77-4da7-426d-b188-176b80d54f8e None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:49.717980 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-185f8e77-4da7-426d-b188-176b80d54f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:49.800578 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1106/2235] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:49.809677 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e03475-dc65-4a5d-b80e-2d66c4e5363a None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:49.810160 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e03475-dc65-4a5d-b80e-2d66c4e5363a None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:49.810507 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e03475-dc65-4a5d-b80e-2d66c4e5363a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:49.836453 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ba6a0-6aa6-4a9b-bdc4-59cdfa45cd01 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:49.842426 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6282e-5938-43a3-b2bb-c74f5e61d0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:49.843433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6282e-5938-43a3-b2bb-c74f5e61d0ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:49.843818 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6282e-5938-43a3-b2bb-c74f5e61d0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:49.844243 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6282e-5938-43a3-b2bb-c74f5e61d0ff admin admin] PATH_INFO: `/v3/users/d1a8dd77d27f4756a438e675b3e98705` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:49.846041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6282e-5938-43a3-b2bb-c74f5e61d0ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1a8dd77d27f4756a438e675b3e98705)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:49.849224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6282e-5938-43a3-b2bb-c74f5e61d0ff admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:49.851730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-d6e03475-dc65-4a5d-b80e-2d66c4e5363a None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:49.853788 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d6e03475-dc65-4a5d-b80e-2d66c4e5363a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:49.864594 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1107/2236] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:58:49.870088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e03475-dc65-4a5d-b80e-2d66c4e5363a None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:49.879134 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:49.884282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e03475-dc65-4a5d-b80e-2d66c4e5363a None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:49.892348 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879ecc6-1f03-437e-99d3-cad19c01a5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:49.893688 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879ecc6-1f03-437e-99d3-cad19c01a5c5 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:49.893841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879ecc6-1f03-437e-99d3-cad19c01a5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:49.894058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879ecc6-1f03-437e-99d3-cad19c01a5c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:49.894951 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879ecc6-1f03-437e-99d3-cad19c01a5c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:58:49.895069 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879ecc6-1f03-437e-99d3-cad19c01a5c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:49.896989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879ecc6-1f03-437e-99d3-cad19c01a5c5 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:49.907141 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:49.917839 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1108/2237] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:58:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:58:49.918345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4cf6282e-5938-43a3-b2bb-c74f5e61d0ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1a8dd77d27f4756a438e675b3e98705'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:49.919720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879ecc6-1f03-437e-99d3-cad19c01a5c5 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:49.935090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879ecc6-1f03-437e-99d3-cad19c01a5c5 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:49.936593 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4cf6282e-5938-43a3-b2bb-c74f5e61d0ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1a8dd77d27f4756a438e675b3e98705'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:49.942112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f6bc91-4391-41fc-b7bd-558a5650aafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:49.942876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6bc91-4391-41fc-b7bd-558a5650aafa admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:49.943286 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6bc91-4391-41fc-b7bd-558a5650aafa admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:49.943499 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6bc91-4391-41fc-b7bd-558a5650aafa admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:49.944352 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f6bc91-4391-41fc-b7bd-558a5650aafa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:49.946561 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4cf6282e-5938-43a3-b2bb-c74f5e61d0ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1a8dd77d27f4756a438e675b3e98705'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:49.947659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f6bc91-4391-41fc-b7bd-558a5650aafa admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:49.948086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4cf6282e-5938-43a3-b2bb-c74f5e61d0ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1a8dd77d27f4756a438e675b3e98705'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:49.949178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4cf6282e-5938-43a3-b2bb-c74f5e61d0ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1a8dd77d27f4756a438e675b3e98705'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:49.973831 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1130/2238] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:58:49 2022] DELETE /identity/v3/users/d1a8dd77d27f4756a438e675b3e98705 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:49.975942 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6282e-5938-43a3-b2bb-c74f5e61d0ff admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:49.978508 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4cf6282e-5938-43a3-b2bb-c74f5e61d0ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:49.983315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1109/2239] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:58:49 2022] POST /identity/v3/projects => generated 367 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:58:49.986808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49015441-4a0c-4d24-a3c3-b7223910f738 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:49.986808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49015441-4a0c-4d24-a3c3-b7223910f738 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:49.986808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49015441-4a0c-4d24-a3c3-b7223910f738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:50.014533 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-49015441-4a0c-4d24-a3c3-b7223910f738 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:50.016281 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-49015441-4a0c-4d24-a3c3-b7223910f738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:50.121283 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6282e-5938-43a3-b2bb-c74f5e61d0ff admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:50.156940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1110/2240] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:50.162512 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49015441-4a0c-4d24-a3c3-b7223910f738 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:50.164097 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-49015441-4a0c-4d24-a3c3-b7223910f738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:50.195805 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49015441-4a0c-4d24-a3c3-b7223910f738 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:50.202214 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10121b65-336e-420d-b26a-05fc51b7e64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:50.203190 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10121b65-336e-420d-b26a-05fc51b7e64b admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:50.203627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10121b65-336e-420d-b26a-05fc51b7e64b admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:50.204213 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10121b65-336e-420d-b26a-05fc51b7e64b admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:50.205133 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10121b65-336e-420d-b26a-05fc51b7e64b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:50.207523 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10121b65-336e-420d-b26a-05fc51b7e64b admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:50.262051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb83b4-e92a-40f4-9555-2a5ed5f207c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:50.262051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb83b4-e92a-40f4-9555-2a5ed5f207c5 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:50.262051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb83b4-e92a-40f4-9555-2a5ed5f207c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:50.262051 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb83b4-e92a-40f4-9555-2a5ed5f207c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:50.262498 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1111/2241] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:58:50 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:50.263460 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10121b65-336e-420d-b26a-05fc51b7e64b admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:50.265065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b9cb83b4-e92a-40f4-9555-2a5ed5f207c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:50.265706 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-10121b65-336e-420d-b26a-05fc51b7e64b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:50.338825 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10121b65-336e-420d-b26a-05fc51b7e64b admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:50.342605 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32859f-8571-4ac8-9a36-0f689afa8559 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:50.348502 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32859f-8571-4ac8-9a36-0f689afa8559 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:50.349010 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32859f-8571-4ac8-9a36-0f689afa8559 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:50.349647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32859f-8571-4ac8-9a36-0f689afa8559 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:50.353056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb83b4-e92a-40f4-9555-2a5ed5f207c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:50.357255 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb83b4-e92a-40f4-9555-2a5ed5f207c5 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:50.364152 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b32859f-8571-4ac8-9a36-0f689afa8559 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:50.365619 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b32859f-8571-4ac8-9a36-0f689afa8559 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:50.380480 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1112/2242] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:50.381492 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32859f-8571-4ac8-9a36-0f689afa8559 service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:50.393227 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32859f-8571-4ac8-9a36-0f689afa8559 service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:50.400348 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f63d02-e0ff-4445-ba67-19904fe20f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:50.401340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f63d02-e0ff-4445-ba67-19904fe20f5a admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:50.401741 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f63d02-e0ff-4445-ba67-19904fe20f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:50.401906 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1131/2243] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:49 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:50.404016 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f63d02-e0ff-4445-ba67-19904fe20f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:50.404016 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f63d02-e0ff-4445-ba67-19904fe20f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:50.404968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f63d02-e0ff-4445-ba67-19904fe20f5a admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:50.417518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1113/2244] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:50.421518 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb83b4-e92a-40f4-9555-2a5ed5f207c5 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:50.447956 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb83b4-e92a-40f4-9555-2a5ed5f207c5 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:50.459431 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2a21e-e24c-4ae0-b4a1-b3df7089ce0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:50.460146 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2a21e-e24c-4ae0-b4a1-b3df7089ce0f admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:50.460420 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2a21e-e24c-4ae0-b4a1-b3df7089ce0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:50.460694 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2a21e-e24c-4ae0-b4a1-b3df7089ce0f admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:50.461557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2a21e-e24c-4ae0-b4a1-b3df7089ce0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:50.463305 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2a21e-e24c-4ae0-b4a1-b3df7089ce0f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:50.481449 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1132/2245] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:58:50.485954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f63d02-e0ff-4445-ba67-19904fe20f5a admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:50.499676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f63d02-e0ff-4445-ba67-19904fe20f5a admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:50.505407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f71683-660a-4d5b-8212-6465edd855f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:50.506446 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f71683-660a-4d5b-8212-6465edd855f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:50.506960 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f71683-660a-4d5b-8212-6465edd855f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:50.507073 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f71683-660a-4d5b-8212-6465edd855f6 admin admin] PATH_INFO: `/v3/projects/b1e038925d3049d5994ec067eb7a014a/users/22840f3ebec549fb8fd2821488fa9f77/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:50.529743 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f71683-660a-4d5b-8212-6465edd855f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1e038925d3049d5994ec067eb7a014a, user_id=22840f3ebec549fb8fd2821488fa9f77, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:50.533187 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f71683-660a-4d5b-8212-6465edd855f6 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:50.555518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1114/2246] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:58:50 2022] PUT /identity/v3/projects/b1e038925d3049d5994ec067eb7a014a/users/22840f3ebec549fb8fd2821488fa9f77/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:58:50.567798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a32ae5-aa5a-4007-8226-0395e307c45f None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:50.568181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a32ae5-aa5a-4007-8226-0395e307c45f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:50.568538 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a32ae5-aa5a-4007-8226-0395e307c45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:50.633138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:50.685403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b9a32ae5-aa5a-4007-8226-0395e307c45f None None] MFA Rules not processed for user `22840f3ebec549fb8fd2821488fa9f77`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:50.693782 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b9a32ae5-aa5a-4007-8226-0395e307c45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:50.790660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1133/2247] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:58:50.795429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f71683-660a-4d5b-8212-6465edd855f6 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:50.811345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a32ae5-aa5a-4007-8226-0395e307c45f None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:50.813021 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b9a32ae5-aa5a-4007-8226-0395e307c45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:50.830146 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:50.884441 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f71683-660a-4d5b-8212-6465edd855f6 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:50.889255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5c2c5-8320-49eb-835c-2f94c8789c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:50.890341 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5c2c5-8320-49eb-835c-2f94c8789c5c admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:50.891153 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5c2c5-8320-49eb-835c-2f94c8789c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:50.891153 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5c2c5-8320-49eb-835c-2f94c8789c5c admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:50.892056 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a5c2c5-8320-49eb-835c-2f94c8789c5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:50.894104 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a5c2c5-8320-49eb-835c-2f94c8789c5c admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:50.931279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a32ae5-aa5a-4007-8226-0395e307c45f None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:50.935884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1115/2248] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:58:50 2022] POST /identity/v3/projects => generated 367 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:58:50.939694 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5c2c5-8320-49eb-835c-2f94c8789c5c admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:50.940749 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a643e0-36d8-4cbe-b00d-5857b79657cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:50.941961 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a643e0-36d8-4cbe-b00d-5857b79657cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:50.942409 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a643e0-36d8-4cbe-b00d-5857b79657cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:50.942683 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a643e0-36d8-4cbe-b00d-5857b79657cc admin admin] PATH_INFO: `/v3/projects/aaaa40cb97c3403f88d72657c5b66215` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:50.944204 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a643e0-36d8-4cbe-b00d-5857b79657cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaaa40cb97c3403f88d72657c5b66215)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:50.946256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a643e0-36d8-4cbe-b00d-5857b79657cc admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.017356 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-48a643e0-36d8-4cbe-b00d-5857b79657cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaaa40cb97c3403f88d72657c5b66215', 'request_id': 'req-48a643e0-36d8-4cbe-b00d-5857b79657cc'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:51.025598 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-48a643e0-36d8-4cbe-b00d-5857b79657cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaaa40cb97c3403f88d72657c5b66215', 'request_id': 'req-48a643e0-36d8-4cbe-b00d-5857b79657cc'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:51.029829 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5c2c5-8320-49eb-835c-2f94c8789c5c admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.036436 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-48a643e0-36d8-4cbe-b00d-5857b79657cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaaa40cb97c3403f88d72657c5b66215', 'request_id': 'req-48a643e0-36d8-4cbe-b00d-5857b79657cc'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:51.054132 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-48a643e0-36d8-4cbe-b00d-5857b79657cc admin admin] The token cache is being invalidate because project aaaa40cb97c3403f88d72657c5b66215 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:51.054436 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-48a643e0-36d8-4cbe-b00d-5857b79657cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:51.055114 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-48a643e0-36d8-4cbe-b00d-5857b79657cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:51.057295 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1134/2249] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:58:50 2022] DELETE /identity/v3/projects/aaaa40cb97c3403f88d72657c5b66215 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:58:51.063837 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aacccbd-5b5f-416a-9f0e-cb6cdfba91a5 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.064121 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aacccbd-5b5f-416a-9f0e-cb6cdfba91a5 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.064401 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aacccbd-5b5f-416a-9f0e-cb6cdfba91a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.098055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-4aacccbd-5b5f-416a-9f0e-cb6cdfba91a5 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:51.100179 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4aacccbd-5b5f-416a-9f0e-cb6cdfba91a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:51.121287 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f237e-6f6d-47f3-88e8-0b42991f5c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.122056 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f237e-6f6d-47f3-88e8-0b42991f5c88 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.122379 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f237e-6f6d-47f3-88e8-0b42991f5c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.122662 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f237e-6f6d-47f3-88e8-0b42991f5c88 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.123302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f237e-6f6d-47f3-88e8-0b42991f5c88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.126090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f237e-6f6d-47f3-88e8-0b42991f5c88 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.150530 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:51.159740 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1116/2250] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:58:50 2022] POST /identity/v3/users => generated 290 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:51.163528 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f237e-6f6d-47f3-88e8-0b42991f5c88 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:51.165396 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-8e1f237e-6f6d-47f3-88e8-0b42991f5c88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:51.181799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f237e-6f6d-47f3-88e8-0b42991f5c88 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.189096 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1a0f7-50c2-46e9-a729-57630fd092a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.189096 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1a0f7-50c2-46e9-a729-57630fd092a0 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.189096 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1a0f7-50c2-46e9-a729-57630fd092a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.189096 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1a0f7-50c2-46e9-a729-57630fd092a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.189997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee1a0f7-50c2-46e9-a729-57630fd092a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.191900 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee1a0f7-50c2-46e9-a729-57630fd092a0 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.200403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1135/2251] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:58:51.207315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1117/2252] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:58:51.208768 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed1082-0b1b-4f61-b560-07aa5d628d39 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.209314 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed1082-0b1b-4f61-b560-07aa5d628d39 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.209781 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed1082-0b1b-4f61-b560-07aa5d628d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.212333 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1a0f7-50c2-46e9-a729-57630fd092a0 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:51.224087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1a0f7-50c2-46e9-a729-57630fd092a0 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.229711 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe1d58-0e1e-4afa-b61f-8530efe5770b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.230728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe1d58-0e1e-4afa-b61f-8530efe5770b admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.231666 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe1d58-0e1e-4afa-b61f-8530efe5770b admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.232048 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe1d58-0e1e-4afa-b61f-8530efe5770b admin admin] PATH_INFO: `/v3/projects/15b7dd45b5e5464e99a306577e3a8d13/users/b795248ee94c4ac58b14c8d8e0d60f37/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.238898 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-eaed1082-0b1b-4f61-b560-07aa5d628d39 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:51.240217 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-eaed1082-0b1b-4f61-b560-07aa5d628d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:51.249300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1136/2253] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:51.251966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe1d58-0e1e-4afa-b61f-8530efe5770b admin admin] RBAC: Authorizing `identity:create_grant(project_id=15b7dd45b5e5464e99a306577e3a8d13, user_id=b795248ee94c4ac58b14c8d8e0d60f37, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.254922 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe1d58-0e1e-4afa-b61f-8530efe5770b admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.255474 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed1082-0b1b-4f61-b560-07aa5d628d39 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:51.272805 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed1082-0b1b-4f61-b560-07aa5d628d39 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.274552 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1118/2254] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:58:51 2022] PUT /identity/v3/projects/15b7dd45b5e5464e99a306577e3a8d13/users/b795248ee94c4ac58b14c8d8e0d60f37/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:51.278377 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe1d58-0e1e-4afa-b61f-8530efe5770b admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:51.291455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:51.374087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe1d58-0e1e-4afa-b61f-8530efe5770b admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.381689 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1da603-291e-4511-b824-ee5ae01add2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.382762 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1da603-291e-4511-b824-ee5ae01add2b admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.383356 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1da603-291e-4511-b824-ee5ae01add2b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.384420 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1da603-291e-4511-b824-ee5ae01add2b admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.384554 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56cbe5-a4f5-442d-98fd-42c71f48bd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.385256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56cbe5-a4f5-442d-98fd-42c71f48bd54 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.385517 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56cbe5-a4f5-442d-98fd-42c71f48bd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.385800 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56cbe5-a4f5-442d-98fd-42c71f48bd54 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.386138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1da603-291e-4511-b824-ee5ae01add2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:58:51.386598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56cbe5-a4f5-442d-98fd-42c71f48bd54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.386757 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1da603-291e-4511-b824-ee5ae01add2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.388268 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56cbe5-a4f5-442d-98fd-42c71f48bd54 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.388580 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1da603-291e-4511-b824-ee5ae01add2b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.400944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1119/2255] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:58:51.404547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56cbe5-a4f5-442d-98fd-42c71f48bd54 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:51.406209 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1137/2256] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:58:51.409181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1da603-291e-4511-b824-ee5ae01add2b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:51.417500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56cbe5-a4f5-442d-98fd-42c71f48bd54 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.420907 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1da603-291e-4511-b824-ee5ae01add2b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.423256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914ec9f-5e60-4bfc-aaba-14c49da3e52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.424203 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914ec9f-5e60-4bfc-aaba-14c49da3e52f admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.424968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914ec9f-5e60-4bfc-aaba-14c49da3e52f admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.425383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914ec9f-5e60-4bfc-aaba-14c49da3e52f admin admin] PATH_INFO: `/v3/projects/15b7dd45b5e5464e99a306577e3a8d13/users/b795248ee94c4ac58b14c8d8e0d60f37/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.427366 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e3072-7c50-445b-a0dc-39059644cde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.428043 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914ec9f-5e60-4bfc-aaba-14c49da3e52f admin admin] RBAC: Authorizing `identity:create_grant(project_id=15b7dd45b5e5464e99a306577e3a8d13, user_id=b795248ee94c4ac58b14c8d8e0d60f37, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.428683 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e3072-7c50-445b-a0dc-39059644cde8 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.429164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e3072-7c50-445b-a0dc-39059644cde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.429768 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e3072-7c50-445b-a0dc-39059644cde8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.431484 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914ec9f-5e60-4bfc-aaba-14c49da3e52f admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.431746 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e3072-7c50-445b-a0dc-39059644cde8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.434053 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e3072-7c50-445b-a0dc-39059644cde8 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.445609 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1120/2257] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:58:51 2022] PUT /identity/v3/projects/15b7dd45b5e5464e99a306577e3a8d13/users/b795248ee94c4ac58b14c8d8e0d60f37/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:51.457616 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641fc28-5ac9-4aab-89a7-56e43e9eac07 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.457903 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641fc28-5ac9-4aab-89a7-56e43e9eac07 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.458209 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641fc28-5ac9-4aab-89a7-56e43e9eac07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.481937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1138/2258] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:58:51 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:58:51.481937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63c720-5544-40e5-a11d-e75e16e07584 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.481937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63c720-5544-40e5-a11d-e75e16e07584 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.481937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63c720-5544-40e5-a11d-e75e16e07584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.492058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-b641fc28-5ac9-4aab-89a7-56e43e9eac07 None None] MFA Rules not processed for user `b795248ee94c4ac58b14c8d8e0d60f37`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:51.493768 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-b641fc28-5ac9-4aab-89a7-56e43e9eac07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:51.550537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-8c63c720-5544-40e5-a11d-e75e16e07584 None None] MFA Rules not processed for user `b2935579c8554bed8b1e211c47548ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:51.551829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8c63c720-5544-40e5-a11d-e75e16e07584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:51.577475 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1121/2259] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:51 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:51.578734 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641fc28-5ac9-4aab-89a7-56e43e9eac07 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:51.632472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:51.635777 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1139/2260] 10.222.0.78 () {58 vars in 1009 bytes} [Wed May 25 14:58:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:51.636677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63c720-5544-40e5-a11d-e75e16e07584 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:51.638619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8c63c720-5544-40e5-a11d-e75e16e07584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:51.684911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641fc28-5ac9-4aab-89a7-56e43e9eac07 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.692779 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc8ec0-fe2b-46d3-81fc-07fee4502d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.693847 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc8ec0-fe2b-46d3-81fc-07fee4502d6d admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.694387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc8ec0-fe2b-46d3-81fc-07fee4502d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.694892 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc8ec0-fe2b-46d3-81fc-07fee4502d6d admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.696731 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc8ec0-fe2b-46d3-81fc-07fee4502d6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.700527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc8ec0-fe2b-46d3-81fc-07fee4502d6d admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.705457 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63c720-5544-40e5-a11d-e75e16e07584 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.709327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0314b-4769-40b4-8242-b48528b6bb49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.709975 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0314b-4769-40b4-8242-b48528b6bb49 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.710293 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0314b-4769-40b4-8242-b48528b6bb49 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.710564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0314b-4769-40b4-8242-b48528b6bb49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.722032 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0314b-4769-40b4-8242-b48528b6bb49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.723497 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0314b-4769-40b4-8242-b48528b6bb49 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.734132 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1140/2261] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:51 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:51.734972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0314b-4769-40b4-8242-b48528b6bb49 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:51.736292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-20f0314b-4769-40b4-8242-b48528b6bb49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:51.741039 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1122/2262] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:58:51 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:51.742188 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc8ec0-fe2b-46d3-81fc-07fee4502d6d admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:51.749260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:51.754483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc8ec0-fe2b-46d3-81fc-07fee4502d6d admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.776512 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575dae5e-e9d9-4aac-ae53-77c05df9fadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.777358 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575dae5e-e9d9-4aac-ae53-77c05df9fadb admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.777736 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575dae5e-e9d9-4aac-ae53-77c05df9fadb admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.778218 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575dae5e-e9d9-4aac-ae53-77c05df9fadb admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.779255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575dae5e-e9d9-4aac-ae53-77c05df9fadb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.781261 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575dae5e-e9d9-4aac-ae53-77c05df9fadb admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.803412 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1123/2263] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:51.805724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575dae5e-e9d9-4aac-ae53-77c05df9fadb admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:51.822404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575dae5e-e9d9-4aac-ae53-77c05df9fadb admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.828904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336355a1-edfe-4db2-bb2f-680afb7a34ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.830125 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336355a1-edfe-4db2-bb2f-680afb7a34ab admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.830543 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336355a1-edfe-4db2-bb2f-680afb7a34ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.830892 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336355a1-edfe-4db2-bb2f-680afb7a34ab admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.831741 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336355a1-edfe-4db2-bb2f-680afb7a34ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.834215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336355a1-edfe-4db2-bb2f-680afb7a34ab admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.857589 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1124/2264] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:58:51.858565 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336355a1-edfe-4db2-bb2f-680afb7a34ab admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:51.879945 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0314b-4769-40b4-8242-b48528b6bb49 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.881776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336355a1-edfe-4db2-bb2f-680afb7a34ab admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.887080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a44e6b-1b1c-4044-8497-864c556a461d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe4a2d106d84fc9ac65c046ba675353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.887861 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a44e6b-1b1c-4044-8497-864c556a461d service swift] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.888251 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a44e6b-1b1c-4044-8497-864c556a461d service swift] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.888596 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a44e6b-1b1c-4044-8497-864c556a461d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.889854 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869b608-1b10-4b22-bc39-9ac91bbf62f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:51.890876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869b608-1b10-4b22-bc39-9ac91bbf62f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.891175 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869b608-1b10-4b22-bc39-9ac91bbf62f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.891441 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869b608-1b10-4b22-bc39-9ac91bbf62f5 admin admin] PATH_INFO: `/v3/projects/173de539832e49108aa84590ac057373/users/a50ab9f456924fb7b4503684f9ec7cc2/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:51.908912 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a44e6b-1b1c-4044-8497-864c556a461d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.908912 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a44e6b-1b1c-4044-8497-864c556a461d service swift] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.925000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869b608-1b10-4b22-bc39-9ac91bbf62f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=173de539832e49108aa84590ac057373, user_id=a50ab9f456924fb7b4503684f9ec7cc2, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:51.928160 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869b608-1b10-4b22-bc39-9ac91bbf62f5 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:51.936736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1141/2265] 10.222.0.78 () {60 vars in 1384 bytes} [Wed May 25 14:58:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 14:58:51.937762 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a44e6b-1b1c-4044-8497-864c556a461d service swift] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:51.950352 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1125/2266] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:58:51 2022] PUT /identity/v3/projects/173de539832e49108aa84590ac057373/users/a50ab9f456924fb7b4503684f9ec7cc2/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:51.960715 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a44e6b-1b1c-4044-8497-864c556a461d service swift] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:51.966682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e7b67-10f8-41b6-81a1-38d20036a0eb None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:51.967012 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e7b67-10f8-41b6-81a1-38d20036a0eb None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:51.967238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e7b67-10f8-41b6-81a1-38d20036a0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:52.026339 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-f83e7b67-10f8-41b6-81a1-38d20036a0eb None None] MFA Rules not processed for user `a50ab9f456924fb7b4503684f9ec7cc2`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:52.027652 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f83e7b67-10f8-41b6-81a1-38d20036a0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:52.085215 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bd909-2988-40f6-9e5b-56a1fa9cc38f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:52.086495 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bd909-2988-40f6-9e5b-56a1fa9cc38f service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:52.086495 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bd909-2988-40f6-9e5b-56a1fa9cc38f service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:52.086682 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bd909-2988-40f6-9e5b-56a1fa9cc38f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:52.089992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ed2bd909-2988-40f6-9e5b-56a1fa9cc38f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:52.146687 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:52.187986 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1126/2267] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:52.220878 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2bd909-2988-40f6-9e5b-56a1fa9cc38f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:52.224352 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2bd909-2988-40f6-9e5b-56a1fa9cc38f service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:52.226924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e7b67-10f8-41b6-81a1-38d20036a0eb None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:52.228525 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f83e7b67-10f8-41b6-81a1-38d20036a0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:52.241323 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1142/2268] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:51 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:52.245665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e7b67-10f8-41b6-81a1-38d20036a0eb None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:52.251630 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675afb5-1684-4d3a-ab26-1e19bcd7c7e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:52.252542 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675afb5-1684-4d3a-ab26-1e19bcd7c7e5 service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:52.253482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675afb5-1684-4d3a-ab26-1e19bcd7c7e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:52.253906 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675afb5-1684-4d3a-ab26-1e19bcd7c7e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:52.279517 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675afb5-1684-4d3a-ab26-1e19bcd7c7e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:52.280982 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675afb5-1684-4d3a-ab26-1e19bcd7c7e5 service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:52.292451 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1127/2269] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:58:52.323552 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bd909-2988-40f6-9e5b-56a1fa9cc38f service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:52.405466 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bd909-2988-40f6-9e5b-56a1fa9cc38f service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:52.412718 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8435f14-8987-4720-88a0-6ea6136f6a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:52.414165 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8435f14-8987-4720-88a0-6ea6136f6a60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:52.414723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8435f14-8987-4720-88a0-6ea6136f6a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:52.415089 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8435f14-8987-4720-88a0-6ea6136f6a60 admin admin] PATH_INFO: `/v3/users/a50ab9f456924fb7b4503684f9ec7cc2` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:52.416461 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8435f14-8987-4720-88a0-6ea6136f6a60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a50ab9f456924fb7b4503684f9ec7cc2)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:52.418509 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8435f14-8987-4720-88a0-6ea6136f6a60 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:52.478461 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a8435f14-8987-4720-88a0-6ea6136f6a60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a50ab9f456924fb7b4503684f9ec7cc2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:52.501540 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a8435f14-8987-4720-88a0-6ea6136f6a60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a50ab9f456924fb7b4503684f9ec7cc2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:52.512188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a8435f14-8987-4720-88a0-6ea6136f6a60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a50ab9f456924fb7b4503684f9ec7cc2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:52.513234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a8435f14-8987-4720-88a0-6ea6136f6a60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a50ab9f456924fb7b4503684f9ec7cc2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:52.514160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a8435f14-8987-4720-88a0-6ea6136f6a60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a50ab9f456924fb7b4503684f9ec7cc2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:52.533197 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1143/2270] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:58:52 2022] DELETE /identity/v3/users/a50ab9f456924fb7b4503684f9ec7cc2 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:52.537990 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcaf523-c3b3-49b3-a8de-d234734e34a4 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:52.538307 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcaf523-c3b3-49b3-a8de-d234734e34a4 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:52.538582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcaf523-c3b3-49b3-a8de-d234734e34a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:52.567782 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-bdcaf523-c3b3-49b3-a8de-d234734e34a4 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:52.569097 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-bdcaf523-c3b3-49b3-a8de-d234734e34a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:52.715233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1128/2271] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:52.725252 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8435f14-8987-4720-88a0-6ea6136f6a60 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:52.727656 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a8435f14-8987-4720-88a0-6ea6136f6a60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:52.796897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fe193-4381-4c9c-8d36-2e8d946adc05 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:52.797202 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fe193-4381-4c9c-8d36-2e8d946adc05 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:52.797458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fe193-4381-4c9c-8d36-2e8d946adc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:52.818871 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-461fe193-4381-4c9c-8d36-2e8d946adc05 None None] MFA Rules not processed for user `b2935579c8554bed8b1e211c47548ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:52.820081 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-461fe193-4381-4c9c-8d36-2e8d946adc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:52.820335 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:52.853924 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8435f14-8987-4720-88a0-6ea6136f6a60 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:52.863554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed73c45b-8784-4578-946d-355cd3cc0df1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:52.864573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73c45b-8784-4578-946d-355cd3cc0df1 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:52.865173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73c45b-8784-4578-946d-355cd3cc0df1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:52.865300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73c45b-8784-4578-946d-355cd3cc0df1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:52.879087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:52.893302 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed73c45b-8784-4578-946d-355cd3cc0df1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:52.906289 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed73c45b-8784-4578-946d-355cd3cc0df1 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:52.916563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1129/2272] 10.222.0.78 () {58 vars in 967 bytes} [Wed May 25 14:58:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:58:52.921436 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1144/2273] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:52.931811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fe193-4381-4c9c-8d36-2e8d946adc05 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:52.934701 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-461fe193-4381-4c9c-8d36-2e8d946adc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:53.089048 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fe193-4381-4c9c-8d36-2e8d946adc05 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:53.097093 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bf70d-40ca-4519-82fd-1813424e9a70 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe4a2d106d84fc9ac65c046ba675353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:53.098267 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bf70d-40ca-4519-82fd-1813424e9a70 service swift] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:53.098551 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bf70d-40ca-4519-82fd-1813424e9a70 service swift] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:53.098832 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bf70d-40ca-4519-82fd-1813424e9a70 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:53.112962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090bf70d-40ca-4519-82fd-1813424e9a70 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:53.115243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090bf70d-40ca-4519-82fd-1813424e9a70 service swift] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:53.126313 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1130/2274] 10.222.0.78 () {60 vars in 1384 bytes} [Wed May 25 14:58:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 14:58:53.303462 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed73c45b-8784-4578-946d-355cd3cc0df1 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:53.307081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ed73c45b-8784-4578-946d-355cd3cc0df1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:53.327556 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed73c45b-8784-4578-946d-355cd3cc0df1 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:53.333695 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6ba9f-8192-40ad-b69e-26a8df9fc2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:53.334755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6ba9f-8192-40ad-b69e-26a8df9fc2b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:53.335180 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6ba9f-8192-40ad-b69e-26a8df9fc2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:53.335557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6ba9f-8192-40ad-b69e-26a8df9fc2b3 admin admin] PATH_INFO: `/v3/projects/173de539832e49108aa84590ac057373` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:53.337829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6ba9f-8192-40ad-b69e-26a8df9fc2b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=173de539832e49108aa84590ac057373)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:53.340677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6ba9f-8192-40ad-b69e-26a8df9fc2b3 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:53.415414 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e620b5-7f67-4beb-be58-da7328d05a16 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:53.415905 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e620b5-7f67-4beb-be58-da7328d05a16 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:53.416309 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e620b5-7f67-4beb-be58-da7328d05a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:53.419224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5db6ba9f-8192-40ad-b69e-26a8df9fc2b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '173de539832e49108aa84590ac057373', 'request_id': 'req-5db6ba9f-8192-40ad-b69e-26a8df9fc2b3'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:53.428050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5db6ba9f-8192-40ad-b69e-26a8df9fc2b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '173de539832e49108aa84590ac057373', 'request_id': 'req-5db6ba9f-8192-40ad-b69e-26a8df9fc2b3'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:53.437970 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5db6ba9f-8192-40ad-b69e-26a8df9fc2b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '173de539832e49108aa84590ac057373', 'request_id': 'req-5db6ba9f-8192-40ad-b69e-26a8df9fc2b3'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:53.443277 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-f4e620b5-7f67-4beb-be58-da7328d05a16 None None] MFA Rules not processed for user `b2935579c8554bed8b1e211c47548ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:53.444636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f4e620b5-7f67-4beb-be58-da7328d05a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:53.458374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5db6ba9f-8192-40ad-b69e-26a8df9fc2b3 admin admin] The token cache is being invalidate because project 173de539832e49108aa84590ac057373 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:53.459284 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5db6ba9f-8192-40ad-b69e-26a8df9fc2b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:53.460744 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5db6ba9f-8192-40ad-b69e-26a8df9fc2b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:53.463846 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1145/2275] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:58:53 2022] DELETE /identity/v3/projects/173de539832e49108aa84590ac057373 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:53.470288 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b98ed1-a33c-4f79-b6ea-3e03d3ffa7ba None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:53.470557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b98ed1-a33c-4f79-b6ea-3e03d3ffa7ba None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:53.470847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b98ed1-a33c-4f79-b6ea-3e03d3ffa7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:53.494627 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-72b98ed1-a33c-4f79-b6ea-3e03d3ffa7ba None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:53.495867 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-72b98ed1-a33c-4f79-b6ea-3e03d3ffa7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:53.537722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1131/2276] 10.222.0.78 () {58 vars in 1009 bytes} [Wed May 25 14:58:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:58:53.553699 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e620b5-7f67-4beb-be58-da7328d05a16 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:53.556237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f4e620b5-7f67-4beb-be58-da7328d05a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:53.565695 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1146/2277] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:53.576251 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b9e5f-7824-433c-958f-28ae792a86e5 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:53.576647 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b9e5f-7824-433c-958f-28ae792a86e5 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:53.577010 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b9e5f-7824-433c-958f-28ae792a86e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:53.598217 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-137b9e5f-7824-433c-958f-28ae792a86e5 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:53.599492 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-137b9e5f-7824-433c-958f-28ae792a86e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:53.608692 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1147/2278] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:53.612995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b9e5f-7824-433c-958f-28ae792a86e5 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:53.622527 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b9e5f-7824-433c-958f-28ae792a86e5 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:53.627657 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5f56f-f5bf-4c56-872e-c517eb7667df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:53.628345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5f56f-f5bf-4c56-872e-c517eb7667df admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:53.628633 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5f56f-f5bf-4c56-872e-c517eb7667df admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:53.628910 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5f56f-f5bf-4c56-872e-c517eb7667df admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:53.629657 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5f56f-f5bf-4c56-872e-c517eb7667df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:58:53.629969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5f56f-f5bf-4c56-872e-c517eb7667df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:53.631678 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5f56f-f5bf-4c56-872e-c517eb7667df admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:53.636831 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e620b5-7f67-4beb-be58-da7328d05a16 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:53.643093 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a79836-6602-47ed-b25e-a1843974a245 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe4a2d106d84fc9ac65c046ba675353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:53.643762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a79836-6602-47ed-b25e-a1843974a245 service swift] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:53.644041 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a79836-6602-47ed-b25e-a1843974a245 service swift] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:53.644315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a79836-6602-47ed-b25e-a1843974a245 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:53.645124 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1148/2279] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:58:53.650072 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5f56f-f5bf-4c56-872e-c517eb7667df admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:53.656633 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a79836-6602-47ed-b25e-a1843974a245 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:53.658871 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a79836-6602-47ed-b25e-a1843974a245 service swift] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:53.662740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5f56f-f5bf-4c56-872e-c517eb7667df admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:53.668253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c4812-078d-48bc-92d1-61fbc0fe5706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:53.669115 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c4812-078d-48bc-92d1-61fbc0fe5706 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:53.669402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c4812-078d-48bc-92d1-61fbc0fe5706 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:53.669681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c4812-078d-48bc-92d1-61fbc0fe5706 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:53.670411 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1132/2280] 10.222.0.78 () {60 vars in 1384 bytes} [Wed May 25 14:58:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 14:58:53.670550 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c4812-078d-48bc-92d1-61fbc0fe5706 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:53.672535 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c4812-078d-48bc-92d1-61fbc0fe5706 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:53.709144 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1149/2281] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:58:53 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:53.710854 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a79836-6602-47ed-b25e-a1843974a245 service swift] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:53.718907 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:53.723652 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a79836-6602-47ed-b25e-a1843974a245 service swift] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:53.805023 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569854e3-d471-4dd6-bfcf-056551d1c285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:53.805023 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569854e3-d471-4dd6-bfcf-056551d1c285 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:53.805023 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569854e3-d471-4dd6-bfcf-056551d1c285 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:53.805023 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569854e3-d471-4dd6-bfcf-056551d1c285 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:53.805895 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569854e3-d471-4dd6-bfcf-056551d1c285 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:53.805895 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569854e3-d471-4dd6-bfcf-056551d1c285 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:53.828043 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1133/2282] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:58:53 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:58:53.832572 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c4812-078d-48bc-92d1-61fbc0fe5706 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:53.846958 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c4812-078d-48bc-92d1-61fbc0fe5706 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:53.854050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d82f3-032e-4fc1-97c2-b2ac21b439b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:53.855096 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d82f3-032e-4fc1-97c2-b2ac21b439b9 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:53.855495 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d82f3-032e-4fc1-97c2-b2ac21b439b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:53.856057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d82f3-032e-4fc1-97c2-b2ac21b439b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:53.856952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d82f3-032e-4fc1-97c2-b2ac21b439b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:53.858826 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d82f3-032e-4fc1-97c2-b2ac21b439b9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:53.873212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1150/2283] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:53.879462 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569854e3-d471-4dd6-bfcf-056551d1c285 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:53.891591 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569854e3-d471-4dd6-bfcf-056551d1c285 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:53.903794 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed5ffc0-d9f0-48cc-9cb1-c84e9f8cee46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:53.906112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5ffc0-d9f0-48cc-9cb1-c84e9f8cee46 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:53.906764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5ffc0-d9f0-48cc-9cb1-c84e9f8cee46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:53.907416 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5ffc0-d9f0-48cc-9cb1-c84e9f8cee46 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:53.908695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed5ffc0-d9f0-48cc-9cb1-c84e9f8cee46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:53.913577 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed5ffc0-d9f0-48cc-9cb1-c84e9f8cee46 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:53.925815 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1134/2284] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:58:53.930148 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d82f3-032e-4fc1-97c2-b2ac21b439b9 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:53.942802 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d82f3-032e-4fc1-97c2-b2ac21b439b9 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:53.955916 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e61cd0-2bf4-4362-ba0e-ba325b7431c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:53.956827 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e61cd0-2bf4-4362-ba0e-ba325b7431c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:53.957110 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e61cd0-2bf4-4362-ba0e-ba325b7431c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:53.957386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e61cd0-2bf4-4362-ba0e-ba325b7431c4 admin admin] PATH_INFO: `/v3/projects/99281bdc8f6340a1be03774c255ae4ba/users/daec78050ee547699910603eff8cb7ca/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:53.980097 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e61cd0-2bf4-4362-ba0e-ba325b7431c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99281bdc8f6340a1be03774c255ae4ba, user_id=daec78050ee547699910603eff8cb7ca, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:53.983574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e61cd0-2bf4-4362-ba0e-ba325b7431c4 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:53.986966 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:54.005318 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1151/2285] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:58:53 2022] PUT /identity/v3/projects/99281bdc8f6340a1be03774c255ae4ba/users/daec78050ee547699910603eff8cb7ca/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:54.011023 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fc9bc-d09a-45f0-b05e-b2251163bdeb None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:54.011311 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fc9bc-d09a-45f0-b05e-b2251163bdeb None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:54.011599 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fc9bc-d09a-45f0-b05e-b2251163bdeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:54.032912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-9b1fc9bc-d09a-45f0-b05e-b2251163bdeb None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:54.034445 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-9b1fc9bc-d09a-45f0-b05e-b2251163bdeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:54.124419 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1135/2286] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:58:54.136861 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e61cd0-2bf4-4362-ba0e-ba325b7431c4 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:54.138896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-03e61cd0-2bf4-4362-ba0e-ba325b7431c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:54.219012 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd235a3b-c859-467f-ba55-1f351bfabadb None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:54.219379 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd235a3b-c859-467f-ba55-1f351bfabadb None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:54.219714 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd235a3b-c859-467f-ba55-1f351bfabadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:54.230753 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e61cd0-2bf4-4362-ba0e-ba325b7431c4 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:54.234587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f028bc3-3209-4681-bf85-652a36e8825f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:54.235253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f028bc3-3209-4681-bf85-652a36e8825f service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:54.235543 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f028bc3-3209-4681-bf85-652a36e8825f service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:54.235822 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f028bc3-3209-4681-bf85-652a36e8825f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:54.240057 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-bd235a3b-c859-467f-ba55-1f351bfabadb None None] MFA Rules not processed for user `b2935579c8554bed8b1e211c47548ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:54.241357 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-bd235a3b-c859-467f-ba55-1f351bfabadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:54.250413 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f028bc3-3209-4681-bf85-652a36e8825f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:54.251903 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f028bc3-3209-4681-bf85-652a36e8825f service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:54.264997 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1152/2287] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:58:54.326946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1136/2288] 10.222.0.78 () {58 vars in 967 bytes} [Wed May 25 14:58:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:58:54.351386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f028bc3-3209-4681-bf85-652a36e8825f service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:54.452074 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f028bc3-3209-4681-bf85-652a36e8825f service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:54.464223 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eff74a-8073-4162-a468-f0ab23c3166e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe4a2d106d84fc9ac65c046ba675353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:54.465779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eff74a-8073-4162-a468-f0ab23c3166e service swift] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:54.466798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eff74a-8073-4162-a468-f0ab23c3166e service swift] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:54.467643 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eff74a-8073-4162-a468-f0ab23c3166e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:54.481681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eff74a-8073-4162-a468-f0ab23c3166e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:54.483981 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eff74a-8073-4162-a468-f0ab23c3166e service swift] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:54.501555 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1153/2289] 10.222.0.78 () {60 vars in 1384 bytes} [Wed May 25 14:58:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 14:58:55.807323 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd235a3b-c859-467f-ba55-1f351bfabadb None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:55.808946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-bd235a3b-c859-467f-ba55-1f351bfabadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:55.835108 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd235a3b-c859-467f-ba55-1f351bfabadb None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:55.852909 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ba5b6-452f-420d-b4b6-02144bd5eb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:55.854361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ba5b6-452f-420d-b4b6-02144bd5eb50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:55.854797 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ba5b6-452f-420d-b4b6-02144bd5eb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:55.855232 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ba5b6-452f-420d-b4b6-02144bd5eb50 admin admin] PATH_INFO: `/v3/users/6afcf92d1eeb476ba0a9b4cd38e68e02` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:55.858614 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ba5b6-452f-420d-b4b6-02144bd5eb50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6afcf92d1eeb476ba0a9b4cd38e68e02)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:55.869282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ba5b6-452f-420d-b4b6-02144bd5eb50 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:55.928006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:55.983886 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-788ba5b6-452f-420d-b4b6-02144bd5eb50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6afcf92d1eeb476ba0a9b4cd38e68e02'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:56.017551 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-788ba5b6-452f-420d-b4b6-02144bd5eb50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6afcf92d1eeb476ba0a9b4cd38e68e02'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:56.038344 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-788ba5b6-452f-420d-b4b6-02144bd5eb50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6afcf92d1eeb476ba0a9b4cd38e68e02'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:56.039960 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-788ba5b6-452f-420d-b4b6-02144bd5eb50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6afcf92d1eeb476ba0a9b4cd38e68e02'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:56.040830 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-788ba5b6-452f-420d-b4b6-02144bd5eb50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6afcf92d1eeb476ba0a9b4cd38e68e02'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:56.073729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1137/2290] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:58:55 2022] DELETE /identity/v3/users/6afcf92d1eeb476ba0a9b4cd38e68e02 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:56.077952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05262a14-2a54-4538-8bbd-82ea9502383e None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:56.078492 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05262a14-2a54-4538-8bbd-82ea9502383e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:56.078587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05262a14-2a54-4538-8bbd-82ea9502383e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:56.122970 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-05262a14-2a54-4538-8bbd-82ea9502383e None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:56.124264 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-05262a14-2a54-4538-8bbd-82ea9502383e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:56.269555 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1154/2291] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:58:56.285559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ba5b6-452f-420d-b4b6-02144bd5eb50 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:56.287491 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-788ba5b6-452f-420d-b4b6-02144bd5eb50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:56.432789 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ba5b6-452f-420d-b4b6-02144bd5eb50 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:56.439248 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1af2c-e7ca-4b30-a25b-7335cbca350c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:56.440216 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1af2c-e7ca-4b30-a25b-7335cbca350c service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:56.440739 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1af2c-e7ca-4b30-a25b-7335cbca350c service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:56.441394 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1af2c-e7ca-4b30-a25b-7335cbca350c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:56.470950 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1af2c-e7ca-4b30-a25b-7335cbca350c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:56.476461 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1af2c-e7ca-4b30-a25b-7335cbca350c service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:56.524719 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1138/2292] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:57.054969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05262a14-2a54-4538-8bbd-82ea9502383e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:57.058142 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-05262a14-2a54-4538-8bbd-82ea9502383e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:57.087171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05262a14-2a54-4538-8bbd-82ea9502383e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:57.098219 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de65f42-e5bb-4198-9bf0-e972d8626392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:57.101454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de65f42-e5bb-4198-9bf0-e972d8626392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:57.102892 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de65f42-e5bb-4198-9bf0-e972d8626392 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:57.103615 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de65f42-e5bb-4198-9bf0-e972d8626392 admin admin] PATH_INFO: `/v3/projects/5d867311a11b4be482ffdfed4731a86f` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:57.106669 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de65f42-e5bb-4198-9bf0-e972d8626392 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d867311a11b4be482ffdfed4731a86f)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:57.114579 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de65f42-e5bb-4198-9bf0-e972d8626392 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:57.179196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:57.233102 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0de65f42-e5bb-4198-9bf0-e972d8626392 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d867311a11b4be482ffdfed4731a86f', 'request_id': 'req-0de65f42-e5bb-4198-9bf0-e972d8626392'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:57.251262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0de65f42-e5bb-4198-9bf0-e972d8626392 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d867311a11b4be482ffdfed4731a86f', 'request_id': 'req-0de65f42-e5bb-4198-9bf0-e972d8626392'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:57.268011 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0de65f42-e5bb-4198-9bf0-e972d8626392 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d867311a11b4be482ffdfed4731a86f', 'request_id': 'req-0de65f42-e5bb-4198-9bf0-e972d8626392'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:57.309605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0de65f42-e5bb-4198-9bf0-e972d8626392 admin admin] The token cache is being invalidate because project 5d867311a11b4be482ffdfed4731a86f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:57.310175 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0de65f42-e5bb-4198-9bf0-e972d8626392 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:57.315746 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0de65f42-e5bb-4198-9bf0-e972d8626392 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:57.326615 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1af2c-e7ca-4b30-a25b-7335cbca350c service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:57.328401 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1155/2293] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:58:57 2022] DELETE /identity/v3/projects/5d867311a11b4be482ffdfed4731a86f => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:57.349610 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1af2c-e7ca-4b30-a25b-7335cbca350c service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:57.483779 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e439167-4784-46a3-9344-eb9bea8ac553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:57.485684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e439167-4784-46a3-9344-eb9bea8ac553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:57.486345 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e439167-4784-46a3-9344-eb9bea8ac553 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:57.486993 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e439167-4784-46a3-9344-eb9bea8ac553 admin admin] PATH_INFO: `/v3/users/64ebc54f27524a3ebc2f1e279c0fbe54` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:57.492316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e439167-4784-46a3-9344-eb9bea8ac553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64ebc54f27524a3ebc2f1e279c0fbe54)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:57.495129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e439167-4784-46a3-9344-eb9bea8ac553 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:57.610792 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:57.613846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-5e439167-4784-46a3-9344-eb9bea8ac553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64ebc54f27524a3ebc2f1e279c0fbe54'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:57.636914 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-5e439167-4784-46a3-9344-eb9bea8ac553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64ebc54f27524a3ebc2f1e279c0fbe54'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:57.650178 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-5e439167-4784-46a3-9344-eb9bea8ac553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64ebc54f27524a3ebc2f1e279c0fbe54'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:57.651515 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-5e439167-4784-46a3-9344-eb9bea8ac553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64ebc54f27524a3ebc2f1e279c0fbe54'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:57.652500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-5e439167-4784-46a3-9344-eb9bea8ac553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64ebc54f27524a3ebc2f1e279c0fbe54'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:57.671534 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1139/2294] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:58:57 2022] DELETE /identity/v3/users/64ebc54f27524a3ebc2f1e279c0fbe54 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:57.878077 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de65f42-e5bb-4198-9bf0-e972d8626392 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:57.881441 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-0de65f42-e5bb-4198-9bf0-e972d8626392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:57.919618 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de65f42-e5bb-4198-9bf0-e972d8626392 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:58.062650 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81883437-255a-4512-acc7-4811316e0d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:58.063813 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81883437-255a-4512-acc7-4811316e0d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:58.064305 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81883437-255a-4512-acc7-4811316e0d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:58.064693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81883437-255a-4512-acc7-4811316e0d70 admin admin] PATH_INFO: `/v3/users/aef8ca7cd35d451ba228a058892f2da4` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:58.066748 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81883437-255a-4512-acc7-4811316e0d70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aef8ca7cd35d451ba228a058892f2da4)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:58.069291 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81883437-255a-4512-acc7-4811316e0d70 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:58.216480 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-81883437-255a-4512-acc7-4811316e0d70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aef8ca7cd35d451ba228a058892f2da4'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:58.248715 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e439167-4784-46a3-9344-eb9bea8ac553 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:58.250866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-5e439167-4784-46a3-9344-eb9bea8ac553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:58.255752 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-81883437-255a-4512-acc7-4811316e0d70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aef8ca7cd35d451ba228a058892f2da4'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:58.272729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e439167-4784-46a3-9344-eb9bea8ac553 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:58.277822 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-81883437-255a-4512-acc7-4811316e0d70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aef8ca7cd35d451ba228a058892f2da4'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:58.278430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-81883437-255a-4512-acc7-4811316e0d70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aef8ca7cd35d451ba228a058892f2da4'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:58.279447 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-81883437-255a-4512-acc7-4811316e0d70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aef8ca7cd35d451ba228a058892f2da4'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:58.295072 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d56b4-e652-4f8a-a90d-aace26e59703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:58.302509 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1156/2295] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:58:57 2022] DELETE /identity/v3/users/aef8ca7cd35d451ba228a058892f2da4 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:58.307686 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d56b4-e652-4f8a-a90d-aace26e59703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:58.307900 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d56b4-e652-4f8a-a90d-aace26e59703 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:58.307985 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5302d36-01ac-427d-a2f5-046d445bf42c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:58.308065 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d56b4-e652-4f8a-a90d-aace26e59703 admin admin] PATH_INFO: `/v3/projects/cea6e5b216414e27890aca6d6176273d` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:58.308185 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5302d36-01ac-427d-a2f5-046d445bf42c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:58.308473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5302d36-01ac-427d-a2f5-046d445bf42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:58.317895 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d56b4-e652-4f8a-a90d-aace26e59703 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cea6e5b216414e27890aca6d6176273d)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:58.322516 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d56b4-e652-4f8a-a90d-aace26e59703 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:58.334696 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-f5302d36-01ac-427d-a2f5-046d445bf42c None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:58.336607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f5302d36-01ac-427d-a2f5-046d445bf42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:58.387849 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:58.436898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6f8d56b4-e652-4f8a-a90d-aace26e59703 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cea6e5b216414e27890aca6d6176273d', 'request_id': 'req-6f8d56b4-e652-4f8a-a90d-aace26e59703'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:58.448077 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1157/2296] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:58:58.449374 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6f8d56b4-e652-4f8a-a90d-aace26e59703 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cea6e5b216414e27890aca6d6176273d', 'request_id': 'req-6f8d56b4-e652-4f8a-a90d-aace26e59703'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:58.452973 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5302d36-01ac-427d-a2f5-046d445bf42c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:58.454519 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f5302d36-01ac-427d-a2f5-046d445bf42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:58.458679 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6f8d56b4-e652-4f8a-a90d-aace26e59703 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cea6e5b216414e27890aca6d6176273d', 'request_id': 'req-6f8d56b4-e652-4f8a-a90d-aace26e59703'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:58.469456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5302d36-01ac-427d-a2f5-046d445bf42c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:58.479429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6f8d56b4-e652-4f8a-a90d-aace26e59703 admin admin] The token cache is being invalidate because project cea6e5b216414e27890aca6d6176273d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:58.479735 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6f8d56b4-e652-4f8a-a90d-aace26e59703 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:58.480502 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6f8d56b4-e652-4f8a-a90d-aace26e59703 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:58.482733 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1140/2297] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:58:58 2022] DELETE /identity/v3/projects/cea6e5b216414e27890aca6d6176273d => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:58:58.483962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d56b4-e652-4f8a-a90d-aace26e59703 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:58.485852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-6f8d56b4-e652-4f8a-a90d-aace26e59703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:58.570428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d56b4-e652-4f8a-a90d-aace26e59703 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:58.577976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4119b-a1d9-4963-91ff-655486c44cae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:58.579095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4119b-a1d9-4963-91ff-655486c44cae service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:58.580942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4119b-a1d9-4963-91ff-655486c44cae service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:58.581414 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4119b-a1d9-4963-91ff-655486c44cae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:58.584112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d4a4119b-a1d9-4963-91ff-655486c44cae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:58.590316 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451f67f-185b-47c5-bf2d-20976f3666b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:58.594259 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451f67f-185b-47c5-bf2d-20976f3666b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:58.594982 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451f67f-185b-47c5-bf2d-20976f3666b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:58.596160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451f67f-185b-47c5-bf2d-20976f3666b9 admin admin] PATH_INFO: `/v3/users/22840f3ebec549fb8fd2821488fa9f77` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:58.601345 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4119b-a1d9-4963-91ff-655486c44cae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:58.601966 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451f67f-185b-47c5-bf2d-20976f3666b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22840f3ebec549fb8fd2821488fa9f77)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:58.604129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4119b-a1d9-4963-91ff-655486c44cae service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:58.605313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451f67f-185b-47c5-bf2d-20976f3666b9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:58.629681 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1141/2298] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:58.648686 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4119b-a1d9-4963-91ff-655486c44cae service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:58.651020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d4a4119b-a1d9-4963-91ff-655486c44cae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:58.675750 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4119b-a1d9-4963-91ff-655486c44cae service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:58.692542 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63c34f-49b0-44f3-ba06-579001bb4fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:58.694748 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63c34f-49b0-44f3-ba06-579001bb4fcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:58.695478 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63c34f-49b0-44f3-ba06-579001bb4fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:58.699835 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63c34f-49b0-44f3-ba06-579001bb4fcc admin admin] PATH_INFO: `/v3/users/b0f4005528044659a8a24ab17b4937e1` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:58.703775 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c63c34f-49b0-44f3-ba06-579001bb4fcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0f4005528044659a8a24ab17b4937e1)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:58.719247 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c63c34f-49b0-44f3-ba06-579001bb4fcc admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:58.746566 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4451f67f-185b-47c5-bf2d-20976f3666b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22840f3ebec549fb8fd2821488fa9f77'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:58.781414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4451f67f-185b-47c5-bf2d-20976f3666b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22840f3ebec549fb8fd2821488fa9f77'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:58.803113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:58.805724 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4451f67f-185b-47c5-bf2d-20976f3666b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22840f3ebec549fb8fd2821488fa9f77'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:58.806626 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4451f67f-185b-47c5-bf2d-20976f3666b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22840f3ebec549fb8fd2821488fa9f77'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:58.807193 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-5c63c34f-49b0-44f3-ba06-579001bb4fcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0f4005528044659a8a24ab17b4937e1'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:58.807580 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-4451f67f-185b-47c5-bf2d-20976f3666b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22840f3ebec549fb8fd2821488fa9f77'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:58.832919 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-5c63c34f-49b0-44f3-ba06-579001bb4fcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0f4005528044659a8a24ab17b4937e1'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:58.850908 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1158/2299] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:58:58 2022] DELETE /identity/v3/users/22840f3ebec549fb8fd2821488fa9f77 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:58.863452 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e2406-9c7e-496b-8a5c-335267d5cc0d None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:58.863752 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e2406-9c7e-496b-8a5c-335267d5cc0d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:58.866962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-5c63c34f-49b0-44f3-ba06-579001bb4fcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0f4005528044659a8a24ab17b4937e1'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:58.867380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e2406-9c7e-496b-8a5c-335267d5cc0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:58.868867 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-5c63c34f-49b0-44f3-ba06-579001bb4fcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0f4005528044659a8a24ab17b4937e1'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:58.869951 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-5c63c34f-49b0-44f3-ba06-579001bb4fcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0f4005528044659a8a24ab17b4937e1'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:58.906314 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1142/2300] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:58:58 2022] DELETE /identity/v3/users/b0f4005528044659a8a24ab17b4937e1 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:58.966239 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-e81e2406-9c7e-496b-8a5c-335267d5cc0d None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:58:58.969002 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e81e2406-9c7e-496b-8a5c-335267d5cc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:59.093834 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:59.122491 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1159/2301] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:58:59.138337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63c34f-49b0-44f3-ba06-579001bb4fcc admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:59.140450 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-5c63c34f-49b0-44f3-ba06-579001bb4fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:59.237788 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e2406-9c7e-496b-8a5c-335267d5cc0d None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:59.239542 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e81e2406-9c7e-496b-8a5c-335267d5cc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:59.262852 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e2406-9c7e-496b-8a5c-335267d5cc0d None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:59.278736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e9f82-10d9-4a96-a8c0-0919e5275c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:59.280460 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e9f82-10d9-4a96-a8c0-0919e5275c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:59.280792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e9f82-10d9-4a96-a8c0-0919e5275c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:59.282559 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e9f82-10d9-4a96-a8c0-0919e5275c6e admin admin] PATH_INFO: `/v3/projects/56174c2dc6ce472697756899661f4838` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:59.284220 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e9f82-10d9-4a96-a8c0-0919e5275c6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=56174c2dc6ce472697756899661f4838)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:59.286212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e9f82-10d9-4a96-a8c0-0919e5275c6e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:59.314187 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63c34f-49b0-44f3-ba06-579001bb4fcc admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:59.324322 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa79098-83d0-45b7-aa17-0ee5a6ba3639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:59.325374 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa79098-83d0-45b7-aa17-0ee5a6ba3639 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:59.325656 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa79098-83d0-45b7-aa17-0ee5a6ba3639 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:59.326238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa79098-83d0-45b7-aa17-0ee5a6ba3639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:59.348767 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa79098-83d0-45b7-aa17-0ee5a6ba3639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:59.358972 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa79098-83d0-45b7-aa17-0ee5a6ba3639 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:59.397033 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1143/2302] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:58:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:59.411089 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c49e9f82-10d9-4a96-a8c0-0919e5275c6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56174c2dc6ce472697756899661f4838', 'request_id': 'req-c49e9f82-10d9-4a96-a8c0-0919e5275c6e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:59.424436 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c49e9f82-10d9-4a96-a8c0-0919e5275c6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56174c2dc6ce472697756899661f4838', 'request_id': 'req-c49e9f82-10d9-4a96-a8c0-0919e5275c6e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:59.439568 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c49e9f82-10d9-4a96-a8c0-0919e5275c6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56174c2dc6ce472697756899661f4838', 'request_id': 'req-c49e9f82-10d9-4a96-a8c0-0919e5275c6e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:59.465609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c49e9f82-10d9-4a96-a8c0-0919e5275c6e admin admin] The token cache is being invalidate because project 56174c2dc6ce472697756899661f4838 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:59.466269 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c49e9f82-10d9-4a96-a8c0-0919e5275c6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:59.467511 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c49e9f82-10d9-4a96-a8c0-0919e5275c6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:59.469976 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1160/2303] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:58:59 2022] DELETE /identity/v3/projects/56174c2dc6ce472697756899661f4838 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:59.553904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa79098-83d0-45b7-aa17-0ee5a6ba3639 service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:58:59.556524 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-0aa79098-83d0-45b7-aa17-0ee5a6ba3639 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:58:59.584112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa79098-83d0-45b7-aa17-0ee5a6ba3639 service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:58:59.771320 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185a715-9bcf-44e0-9533-a422d4949878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:58:59.772366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185a715-9bcf-44e0-9533-a422d4949878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:59.772654 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185a715-9bcf-44e0-9533-a422d4949878 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:59.772927 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185a715-9bcf-44e0-9533-a422d4949878 admin admin] PATH_INFO: `/v3/projects/0052632cfce148b3942fd1cd6cf92218` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:58:59.774422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185a715-9bcf-44e0-9533-a422d4949878 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0052632cfce148b3942fd1cd6cf92218)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:58:59.776575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185a715-9bcf-44e0-9533-a422d4949878 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:58:59.854133 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-4185a715-9bcf-44e0-9533-a422d4949878 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0052632cfce148b3942fd1cd6cf92218', 'request_id': 'req-4185a715-9bcf-44e0-9533-a422d4949878'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:59.870030 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-4185a715-9bcf-44e0-9533-a422d4949878 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0052632cfce148b3942fd1cd6cf92218', 'request_id': 'req-4185a715-9bcf-44e0-9533-a422d4949878'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:59.878212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-4185a715-9bcf-44e0-9533-a422d4949878 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0052632cfce148b3942fd1cd6cf92218', 'request_id': 'req-4185a715-9bcf-44e0-9533-a422d4949878'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:59.891010 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:58:59.926255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-4185a715-9bcf-44e0-9533-a422d4949878 admin admin] The token cache is being invalidate because project 0052632cfce148b3942fd1cd6cf92218 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:58:59.926830 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-4185a715-9bcf-44e0-9533-a422d4949878 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:59.928757 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-4185a715-9bcf-44e0-9533-a422d4949878 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:58:59.937252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1144/2304] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:58:59 2022] DELETE /identity/v3/projects/0052632cfce148b3942fd1cd6cf92218 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:58:59.939595 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a3b6f-0d09-4620-a102-58047d26b097 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:58:59.940243 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a3b6f-0d09-4620-a102-58047d26b097 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:58:59.940654 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a3b6f-0d09-4620-a102-58047d26b097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:00.010084 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-ec1a3b6f-0d09-4620-a102-58047d26b097 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:00.017760 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ec1a3b6f-0d09-4620-a102-58047d26b097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:00.040536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185a715-9bcf-44e0-9533-a422d4949878 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:00.042623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-4185a715-9bcf-44e0-9533-a422d4949878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:00.190948 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185a715-9bcf-44e0-9533-a422d4949878 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:00.199310 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb87d8-4428-4318-9c79-3f1be0f77dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:00.200451 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb87d8-4428-4318-9c79-3f1be0f77dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:00.200660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb87d8-4428-4318-9c79-3f1be0f77dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:00.200945 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb87d8-4428-4318-9c79-3f1be0f77dd7 admin admin] PATH_INFO: `/v3/projects/b1e038925d3049d5994ec067eb7a014a` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:00.202620 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cb87d8-4428-4318-9c79-3f1be0f77dd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1e038925d3049d5994ec067eb7a014a)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:00.204964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cb87d8-4428-4318-9c79-3f1be0f77dd7 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:00.272291 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1161/2305] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:58:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:00.279671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09beda5-1b72-43d8-aadd-6a45eee015b2 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:00.280089 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09beda5-1b72-43d8-aadd-6a45eee015b2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:00.280442 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09beda5-1b72-43d8-aadd-6a45eee015b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:00.296579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c5cb87d8-4428-4318-9c79-3f1be0f77dd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1e038925d3049d5994ec067eb7a014a', 'request_id': 'req-c5cb87d8-4428-4318-9c79-3f1be0f77dd7'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:00.311124 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c5cb87d8-4428-4318-9c79-3f1be0f77dd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1e038925d3049d5994ec067eb7a014a', 'request_id': 'req-c5cb87d8-4428-4318-9c79-3f1be0f77dd7'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:00.326968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c5cb87d8-4428-4318-9c79-3f1be0f77dd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1e038925d3049d5994ec067eb7a014a', 'request_id': 'req-c5cb87d8-4428-4318-9c79-3f1be0f77dd7'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:00.329038 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-a09beda5-1b72-43d8-aadd-6a45eee015b2 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:00.332434 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a09beda5-1b72-43d8-aadd-6a45eee015b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:00.355202 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c5cb87d8-4428-4318-9c79-3f1be0f77dd7 admin admin] The token cache is being invalidate because project b1e038925d3049d5994ec067eb7a014a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:59:00.355507 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c5cb87d8-4428-4318-9c79-3f1be0f77dd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:00.357460 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c5cb87d8-4428-4318-9c79-3f1be0f77dd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:00.364131 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1145/2306] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:59:00 2022] DELETE /identity/v3/projects/b1e038925d3049d5994ec067eb7a014a => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:00.366100 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb87d8-4428-4318-9c79-3f1be0f77dd7 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:00.368524 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-c5cb87d8-4428-4318-9c79-3f1be0f77dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:00.477813 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1162/2307] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:59:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:00.484672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09beda5-1b72-43d8-aadd-6a45eee015b2 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:00.493729 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb87d8-4428-4318-9c79-3f1be0f77dd7 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:00.500353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96d3e2-01e3-4f7b-bd16-1f80fc38c92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:00.501359 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96d3e2-01e3-4f7b-bd16-1f80fc38c92a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:00.501750 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96d3e2-01e3-4f7b-bd16-1f80fc38c92a admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:00.502167 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96d3e2-01e3-4f7b-bd16-1f80fc38c92a admin admin] PATH_INFO: `/v3/users/b795248ee94c4ac58b14c8d8e0d60f37` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:00.503603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96d3e2-01e3-4f7b-bd16-1f80fc38c92a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b795248ee94c4ac58b14c8d8e0d60f37)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:00.505926 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96d3e2-01e3-4f7b-bd16-1f80fc38c92a admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:00.522522 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09beda5-1b72-43d8-aadd-6a45eee015b2 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:00.542800 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d6a67-9076-477b-9679-db9f4ed4786f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:00.543647 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d6a67-9076-477b-9679-db9f4ed4786f admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:00.543937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d6a67-9076-477b-9679-db9f4ed4786f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:00.544232 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d6a67-9076-477b-9679-db9f4ed4786f admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:00.544952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d6a67-9076-477b-9679-db9f4ed4786f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:59:00.545261 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d6a67-9076-477b-9679-db9f4ed4786f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:00.547661 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d6a67-9076-477b-9679-db9f4ed4786f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:00.566314 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1163/2308] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:59:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:59:00.570962 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d6a67-9076-477b-9679-db9f4ed4786f admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:00.577932 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:00.586352 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:00.586965 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d6a67-9076-477b-9679-db9f4ed4786f admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:00.607585 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad675b9-7c4a-46cc-b65f-e6dc46244f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:00.608636 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad675b9-7c4a-46cc-b65f-e6dc46244f3f admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:00.610592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad675b9-7c4a-46cc-b65f-e6dc46244f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:00.612313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad675b9-7c4a-46cc-b65f-e6dc46244f3f admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:00.613308 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad675b9-7c4a-46cc-b65f-e6dc46244f3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:00.621993 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad675b9-7c4a-46cc-b65f-e6dc46244f3f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:00.626337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-db96d3e2-01e3-4f7b-bd16-1f80fc38c92a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b795248ee94c4ac58b14c8d8e0d60f37'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:00.647483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-db96d3e2-01e3-4f7b-bd16-1f80fc38c92a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b795248ee94c4ac58b14c8d8e0d60f37'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:00.665772 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-db96d3e2-01e3-4f7b-bd16-1f80fc38c92a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b795248ee94c4ac58b14c8d8e0d60f37'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:00.666447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-db96d3e2-01e3-4f7b-bd16-1f80fc38c92a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b795248ee94c4ac58b14c8d8e0d60f37'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:00.668377 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-db96d3e2-01e3-4f7b-bd16-1f80fc38c92a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b795248ee94c4ac58b14c8d8e0d60f37'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:00.677411 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1164/2309] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:59:00 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:59:00.678619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad675b9-7c4a-46cc-b65f-e6dc46244f3f admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:00.682285 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1ad675b9-7c4a-46cc-b65f-e6dc46244f3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:00.695890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1146/2310] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:59:00 2022] DELETE /identity/v3/users/b795248ee94c4ac58b14c8d8e0d60f37 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:00.831762 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad675b9-7c4a-46cc-b65f-e6dc46244f3f admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:00.932094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347dc950-3934-43b2-b503-ecf26e73ba3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:00.933339 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347dc950-3934-43b2-b503-ecf26e73ba3f admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:00.933339 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347dc950-3934-43b2-b503-ecf26e73ba3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:00.933625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347dc950-3934-43b2-b503-ecf26e73ba3f admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:00.934161 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347dc950-3934-43b2-b503-ecf26e73ba3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:00.936250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347dc950-3934-43b2-b503-ecf26e73ba3f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:00.971652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1165/2311] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:59:00 2022] POST /identity/v3/users => generated 284 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:59:00.971936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96d3e2-01e3-4f7b-bd16-1f80fc38c92a admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:00.988003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96d3e2-01e3-4f7b-bd16-1f80fc38c92a admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:01.016455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb3862c-877a-4a24-bc14-c91127cd7bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:01.017589 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3862c-877a-4a24-bc14-c91127cd7bdb admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:01.018220 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3862c-877a-4a24-bc14-c91127cd7bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:01.018841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3862c-877a-4a24-bc14-c91127cd7bdb admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:01.020272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb3862c-877a-4a24-bc14-c91127cd7bdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:01.028432 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb3862c-877a-4a24-bc14-c91127cd7bdb admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:01.059506 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1147/2312] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:59:01.065683 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347dc950-3934-43b2-b503-ecf26e73ba3f admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:01.100699 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347dc950-3934-43b2-b503-ecf26e73ba3f admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:01.110605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98faef8d-9759-4677-84fe-fe02ad34fc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:01.112811 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98faef8d-9759-4677-84fe-fe02ad34fc44 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:01.112811 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98faef8d-9759-4677-84fe-fe02ad34fc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:01.112811 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98faef8d-9759-4677-84fe-fe02ad34fc44 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:01.125808 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98faef8d-9759-4677-84fe-fe02ad34fc44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:01.136071 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98faef8d-9759-4677-84fe-fe02ad34fc44 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:01.178105 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1166/2313] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:59:01.181896 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb3862c-877a-4a24-bc14-c91127cd7bdb admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:01.195540 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb3862c-877a-4a24-bc14-c91127cd7bdb admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:01.222160 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96501eed-cc12-46f3-8c41-d5dff9972329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:01.223887 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96501eed-cc12-46f3-8c41-d5dff9972329 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:01.224410 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96501eed-cc12-46f3-8c41-d5dff9972329 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:01.224901 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96501eed-cc12-46f3-8c41-d5dff9972329 admin admin] PATH_INFO: `/v3/projects/7e6a9b5901d94962a7df33f91ed76715/users/8a7c58d2cd0347fcb07725493c9b3316/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:01.242382 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98faef8d-9759-4677-84fe-fe02ad34fc44 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:01.244820 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-98faef8d-9759-4677-84fe-fe02ad34fc44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:01.268469 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98faef8d-9759-4677-84fe-fe02ad34fc44 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:01.270273 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96501eed-cc12-46f3-8c41-d5dff9972329 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e6a9b5901d94962a7df33f91ed76715, user_id=8a7c58d2cd0347fcb07725493c9b3316, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:01.275807 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96501eed-cc12-46f3-8c41-d5dff9972329 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:01.291862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15762f9-46c5-43dd-9e19-4895958caea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:01.293109 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15762f9-46c5-43dd-9e19-4895958caea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:01.293459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15762f9-46c5-43dd-9e19-4895958caea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:01.293963 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15762f9-46c5-43dd-9e19-4895958caea7 admin admin] PATH_INFO: `/v3/projects/15b7dd45b5e5464e99a306577e3a8d13` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:01.298129 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15762f9-46c5-43dd-9e19-4895958caea7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15b7dd45b5e5464e99a306577e3a8d13)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:01.310441 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15762f9-46c5-43dd-9e19-4895958caea7 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:01.325698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1148/2314] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:59:01 2022] PUT /identity/v3/projects/7e6a9b5901d94962a7df33f91ed76715/users/8a7c58d2cd0347fcb07725493c9b3316/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:01.351069 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af779f-304d-465b-801a-584697253ec0 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:01.351069 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af779f-304d-465b-801a-584697253ec0 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:01.351069 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af779f-304d-465b-801a-584697253ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:01.421646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-e7af779f-304d-465b-801a-584697253ec0 None None] MFA Rules not processed for user `8a7c58d2cd0347fcb07725493c9b3316`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:01.423091 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-e7af779f-304d-465b-801a-584697253ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:01.471176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e15762f9-46c5-43dd-9e19-4895958caea7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15b7dd45b5e5464e99a306577e3a8d13', 'request_id': 'req-e15762f9-46c5-43dd-9e19-4895958caea7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:01.481838 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e15762f9-46c5-43dd-9e19-4895958caea7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15b7dd45b5e5464e99a306577e3a8d13', 'request_id': 'req-e15762f9-46c5-43dd-9e19-4895958caea7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:01.486130 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:01.505031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e15762f9-46c5-43dd-9e19-4895958caea7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15b7dd45b5e5464e99a306577e3a8d13', 'request_id': 'req-e15762f9-46c5-43dd-9e19-4895958caea7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:01.534289 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e15762f9-46c5-43dd-9e19-4895958caea7 admin admin] The token cache is being invalidate because project 15b7dd45b5e5464e99a306577e3a8d13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:59:01.535431 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e15762f9-46c5-43dd-9e19-4895958caea7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:01.538588 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e15762f9-46c5-43dd-9e19-4895958caea7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:01.546292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1167/2315] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:59:01 2022] DELETE /identity/v3/projects/15b7dd45b5e5464e99a306577e3a8d13 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:59:01.549102 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d35d5-84a0-4e97-9d4e-232b0895800d None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:01.549636 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d35d5-84a0-4e97-9d4e-232b0895800d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:01.550210 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d35d5-84a0-4e97-9d4e-232b0895800d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:01.562785 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1149/2316] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:59:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:01.575318 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af779f-304d-465b-801a-584697253ec0 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:01.576850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-e7af779f-304d-465b-801a-584697253ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:01.599175 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-353d35d5-84a0-4e97-9d4e-232b0895800d None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:01.601300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-353d35d5-84a0-4e97-9d4e-232b0895800d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:01.702072 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1168/2317] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:59:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:01.713753 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47853c1a-081e-4b77-8c62-14851384a498 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:01.717840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47853c1a-081e-4b77-8c62-14851384a498 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:01.718609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47853c1a-081e-4b77-8c62-14851384a498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:01.724336 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af779f-304d-465b-801a-584697253ec0 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:01.737210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:01.751724 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-47853c1a-081e-4b77-8c62-14851384a498 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:01.753539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-47853c1a-081e-4b77-8c62-14851384a498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:01.766588 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1169/2318] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:59:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:01.768201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d0322-f878-408b-a223-e86a44d2b5b6 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:01.768567 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d0322-f878-408b-a223-e86a44d2b5b6 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:01.768936 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d0322-f878-408b-a223-e86a44d2b5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:01.799836 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd3d51-1496-41ed-9215-f13b177a41ea service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88cbd76f70d447c4ba92777b2c09821d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:01.800482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd3d51-1496-41ed-9215-f13b177a41ea service glance] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:01.800681 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd3d51-1496-41ed-9215-f13b177a41ea service glance] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:01.800951 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd3d51-1496-41ed-9215-f13b177a41ea service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:01.801851 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-334d0322-f878-408b-a223-e86a44d2b5b6 None None] MFA Rules not processed for user `daec78050ee547699910603eff8cb7ca`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:01.803687 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-334d0322-f878-408b-a223-e86a44d2b5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:01.804509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-dddd3d51-1496-41ed-9215-f13b177a41ea service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:01.893661 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd3d51-1496-41ed-9215-f13b177a41ea service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:01.900649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd3d51-1496-41ed-9215-f13b177a41ea service glance] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:01.913020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1150/2319] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:59:01 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:59:01.915340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd3d51-1496-41ed-9215-f13b177a41ea service glance] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:01.925066 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1170/2320] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:59:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:01.929621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d0322-f878-408b-a223-e86a44d2b5b6 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:01.931716 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-334d0322-f878-408b-a223-e86a44d2b5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:01.935298 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd3d51-1496-41ed-9215-f13b177a41ea service glance] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:01.943681 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2c9b28-2a15-47be-8122-25f0050908d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:01.946426 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c9b28-2a15-47be-8122-25f0050908d5 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:01.946961 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d0322-f878-408b-a223-e86a44d2b5b6 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:01.947083 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c9b28-2a15-47be-8122-25f0050908d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:01.947346 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c9b28-2a15-47be-8122-25f0050908d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:01.948774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2c9b28-2a15-47be-8122-25f0050908d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:59:01.950928 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2c9b28-2a15-47be-8122-25f0050908d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:01.952854 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2c9b28-2a15-47be-8122-25f0050908d5 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:01.957071 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc126b-734f-40a1-b876-6135e4b60e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:01.957752 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc126b-734f-40a1-b876-6135e4b60e21 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:01.958059 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc126b-734f-40a1-b876-6135e4b60e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:01.958386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc126b-734f-40a1-b876-6135e4b60e21 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:01.959093 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc126b-734f-40a1-b876-6135e4b60e21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:01.961120 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc126b-734f-40a1-b876-6135e4b60e21 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:01.969687 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1151/2321] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:59:01.974862 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2c9b28-2a15-47be-8122-25f0050908d5 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:01.990589 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2c9b28-2a15-47be-8122-25f0050908d5 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:01.996681 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262349b-2849-4beb-bd0d-d26dab1d9bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:01.996992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1171/2322] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:59:01 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:59:01.998395 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262349b-2849-4beb-bd0d-d26dab1d9bdb admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:01.998683 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262349b-2849-4beb-bd0d-d26dab1d9bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:01.998966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262349b-2849-4beb-bd0d-d26dab1d9bdb admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:02.000020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262349b-2849-4beb-bd0d-d26dab1d9bdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:02.002055 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262349b-2849-4beb-bd0d-d26dab1d9bdb admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:02.011616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc126b-734f-40a1-b876-6135e4b60e21 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:02.036541 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1152/2323] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:59:01 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:59:02.047274 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262349b-2849-4beb-bd0d-d26dab1d9bdb admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:02.092440 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:02.108818 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc126b-734f-40a1-b876-6135e4b60e21 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:02.131785 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262349b-2849-4beb-bd0d-d26dab1d9bdb admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:02.142271 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acef16-cf37-49c8-8659-38ef51029b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:02.143084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acef16-cf37-49c8-8659-38ef51029b55 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:02.143375 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acef16-cf37-49c8-8659-38ef51029b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:02.143779 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acef16-cf37-49c8-8659-38ef51029b55 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:02.144770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acef16-cf37-49c8-8659-38ef51029b55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:02.146861 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acef16-cf37-49c8-8659-38ef51029b55 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:02.171956 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1153/2324] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:59:02 2022] POST /identity/v3/users => generated 291 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:59:02.179104 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acef16-cf37-49c8-8659-38ef51029b55 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:02.193660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acef16-cf37-49c8-8659-38ef51029b55 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:02.200695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9116a-2393-4dac-a2eb-036911297a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:02.202110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9116a-2393-4dac-a2eb-036911297a81 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:02.202110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9116a-2393-4dac-a2eb-036911297a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:02.202110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9116a-2393-4dac-a2eb-036911297a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:02.202935 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9116a-2393-4dac-a2eb-036911297a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:02.205197 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9116a-2393-4dac-a2eb-036911297a81 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:02.218992 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1154/2325] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:59:02.223319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9116a-2393-4dac-a2eb-036911297a81 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:02.235710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9116a-2393-4dac-a2eb-036911297a81 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:02.236317 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048160f-5fe8-47a3-a743-6bddcdac158b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:02.237177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048160f-5fe8-47a3-a743-6bddcdac158b admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:02.237623 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048160f-5fe8-47a3-a743-6bddcdac158b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:02.237960 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048160f-5fe8-47a3-a743-6bddcdac158b admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:02.238653 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3048160f-5fe8-47a3-a743-6bddcdac158b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:02.240960 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3048160f-5fe8-47a3-a743-6bddcdac158b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:02.241352 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24dd092-296f-4c2e-b5cb-61508d68a572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:02.242257 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dd092-296f-4c2e-b5cb-61508d68a572 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:02.243270 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dd092-296f-4c2e-b5cb-61508d68a572 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:02.243270 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dd092-296f-4c2e-b5cb-61508d68a572 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:02.243759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24dd092-296f-4c2e-b5cb-61508d68a572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:02.245505 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24dd092-296f-4c2e-b5cb-61508d68a572 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:02.252769 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:02.264481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1155/2326] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:59:02.265912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24dd092-296f-4c2e-b5cb-61508d68a572 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:02.274197 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1172/2327] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:59:02 2022] POST /identity/v3/users => generated 295 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:59:02.278421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048160f-5fe8-47a3-a743-6bddcdac158b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:02.281546 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24dd092-296f-4c2e-b5cb-61508d68a572 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:02.292913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52146c-41e9-4879-8d31-09a151115cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:02.293143 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048160f-5fe8-47a3-a743-6bddcdac158b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:02.294355 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52146c-41e9-4879-8d31-09a151115cd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:02.294355 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52146c-41e9-4879-8d31-09a151115cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:02.295013 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52146c-41e9-4879-8d31-09a151115cd0 admin admin] PATH_INFO: `/v3/projects/1650e51a164f4365a5478ae2909017c7/users/cf8570837703471fbe7e95225c9c8e87/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:02.299840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1087fcb-8f0c-4611-ba42-02edf02ae9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:02.301224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1087fcb-8f0c-4611-ba42-02edf02ae9be admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:02.301224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1087fcb-8f0c-4611-ba42-02edf02ae9be admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:02.301224 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1087fcb-8f0c-4611-ba42-02edf02ae9be admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:02.301839 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1087fcb-8f0c-4611-ba42-02edf02ae9be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:02.303539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1087fcb-8f0c-4611-ba42-02edf02ae9be admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:02.313363 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52146c-41e9-4879-8d31-09a151115cd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1650e51a164f4365a5478ae2909017c7, user_id=cf8570837703471fbe7e95225c9c8e87, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:02.316265 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52146c-41e9-4879-8d31-09a151115cd0 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:02.317699 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1173/2328] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:59:02.322882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1087fcb-8f0c-4611-ba42-02edf02ae9be admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:02.337007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1156/2329] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:59:02 2022] PUT /identity/v3/projects/1650e51a164f4365a5478ae2909017c7/users/cf8570837703471fbe7e95225c9c8e87/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:02.347776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb257e5-0ff9-4244-8da5-b6c8bea75d76 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:02.348010 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb257e5-0ff9-4244-8da5-b6c8bea75d76 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:02.348271 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb257e5-0ff9-4244-8da5-b6c8bea75d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:02.350209 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1087fcb-8f0c-4611-ba42-02edf02ae9be admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:02.399769 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-7fb257e5-0ff9-4244-8da5-b6c8bea75d76 None None] MFA Rules not processed for user `cf8570837703471fbe7e95225c9c8e87`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:02.401373 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-7fb257e5-0ff9-4244-8da5-b6c8bea75d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:02.474965 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18074c-90f0-4ad6-8c1c-cb6bde64c246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:02.475617 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18074c-90f0-4ad6-8c1c-cb6bde64c246 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:02.475920 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18074c-90f0-4ad6-8c1c-cb6bde64c246 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:02.476188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18074c-90f0-4ad6-8c1c-cb6bde64c246 admin admin] PATH_INFO: `/v3/projects/9ce506a9ffd44d5fbe080b9fb2f2b693/users/3712ad1188fe405b9b25d3f1254d6746/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:02.508061 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd18074c-90f0-4ad6-8c1c-cb6bde64c246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ce506a9ffd44d5fbe080b9fb2f2b693, user_id=3712ad1188fe405b9b25d3f1254d6746, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:02.512855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd18074c-90f0-4ad6-8c1c-cb6bde64c246 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:02.546496 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1157/2331] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:59:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:59:02.551271 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1174/2331] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:59:02 2022] PUT /identity/v3/projects/9ce506a9ffd44d5fbe080b9fb2f2b693/users/3712ad1188fe405b9b25d3f1254d6746/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:59:02.557047 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb257e5-0ff9-4244-8da5-b6c8bea75d76 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:02.561697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18074c-90f0-4ad6-8c1c-cb6bde64c246 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:02.563778 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-cd18074c-90f0-4ad6-8c1c-cb6bde64c246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:02.577689 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb257e5-0ff9-4244-8da5-b6c8bea75d76 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:02.693460 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18074c-90f0-4ad6-8c1c-cb6bde64c246 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:02.697957 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8004627-914f-4d6b-9301-7511e33023f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:02.698713 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8004627-914f-4d6b-9301-7511e33023f8 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:02.699004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8004627-914f-4d6b-9301-7511e33023f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:02.699294 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8004627-914f-4d6b-9301-7511e33023f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:02.700875 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86966d02-1962-4ca2-a04a-d53cb51b1cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:02.701721 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86966d02-1962-4ca2-a04a-d53cb51b1cce admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:02.702356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86966d02-1962-4ca2-a04a-d53cb51b1cce admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:02.702795 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86966d02-1962-4ca2-a04a-d53cb51b1cce admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:02.703739 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86966d02-1962-4ca2-a04a-d53cb51b1cce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:02.706408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86966d02-1962-4ca2-a04a-d53cb51b1cce admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:02.713222 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:02.731923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1158/2332] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:59:02.736517 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86966d02-1962-4ca2-a04a-d53cb51b1cce admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:02.760089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86966d02-1962-4ca2-a04a-d53cb51b1cce admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:02.769107 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032e36d-b680-4ef7-961a-57e52ea9da14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:02.770383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032e36d-b680-4ef7-961a-57e52ea9da14 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:02.770823 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032e36d-b680-4ef7-961a-57e52ea9da14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:02.771279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032e36d-b680-4ef7-961a-57e52ea9da14 admin admin] PATH_INFO: `/v3/projects/9ce506a9ffd44d5fbe080b9fb2f2b693/users/3712ad1188fe405b9b25d3f1254d6746/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:02.774161 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032e36d-b680-4ef7-961a-57e52ea9da14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ce506a9ffd44d5fbe080b9fb2f2b693, user_id=3712ad1188fe405b9b25d3f1254d6746, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:02.777296 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032e36d-b680-4ef7-961a-57e52ea9da14 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:02.788031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8004627-914f-4d6b-9301-7511e33023f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:02.790788 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1159/2333] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:59:02 2022] PUT /identity/v3/projects/9ce506a9ffd44d5fbe080b9fb2f2b693/users/3712ad1188fe405b9b25d3f1254d6746/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:02.791209 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8004627-914f-4d6b-9301-7511e33023f8 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:02.807916 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1175/2334] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:59:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:02.829694 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032e36d-b680-4ef7-961a-57e52ea9da14 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:02.919892 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:02.940076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032e36d-b680-4ef7-961a-57e52ea9da14 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:02.947817 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86dc413-7db1-4787-957c-f6870dfd8a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:02.949066 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86dc413-7db1-4787-957c-f6870dfd8a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:02.949516 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86dc413-7db1-4787-957c-f6870dfd8a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:02.949948 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86dc413-7db1-4787-957c-f6870dfd8a2d admin admin] PATH_INFO: `/v3/users/cf8570837703471fbe7e95225c9c8e87` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:02.951471 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86dc413-7db1-4787-957c-f6870dfd8a2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf8570837703471fbe7e95225c9c8e87)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:02.953739 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86dc413-7db1-4787-957c-f6870dfd8a2d admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:03.049037 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d86dc413-7db1-4787-957c-f6870dfd8a2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf8570837703471fbe7e95225c9c8e87'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:03.066414 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d86dc413-7db1-4787-957c-f6870dfd8a2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf8570837703471fbe7e95225c9c8e87'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:03.081386 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d86dc413-7db1-4787-957c-f6870dfd8a2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf8570837703471fbe7e95225c9c8e87'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:03.083128 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d86dc413-7db1-4787-957c-f6870dfd8a2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf8570837703471fbe7e95225c9c8e87'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:03.083549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-d86dc413-7db1-4787-957c-f6870dfd8a2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf8570837703471fbe7e95225c9c8e87'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:03.104139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1160/2335] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:59:02 2022] DELETE /identity/v3/users/cf8570837703471fbe7e95225c9c8e87 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:03.109848 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f71637-60e9-4311-8bfc-2cab3c07de5e None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:03.110314 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f71637-60e9-4311-8bfc-2cab3c07de5e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:03.110698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f71637-60e9-4311-8bfc-2cab3c07de5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:03.131681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-39f71637-60e9-4311-8bfc-2cab3c07de5e None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:03.133033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-39f71637-60e9-4311-8bfc-2cab3c07de5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:03.246778 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1176/2336] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:59:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:03.255312 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86dc413-7db1-4787-957c-f6870dfd8a2d admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:03.256915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d86dc413-7db1-4787-957c-f6870dfd8a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:03.340912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86dc413-7db1-4787-957c-f6870dfd8a2d admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:03.346589 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc93a0-5837-4f2a-bbe6-6d3dc250b83d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:03.348138 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc93a0-5837-4f2a-bbe6-6d3dc250b83d service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:03.348138 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc93a0-5837-4f2a-bbe6-6d3dc250b83d service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:03.348138 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc93a0-5837-4f2a-bbe6-6d3dc250b83d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:03.363074 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc93a0-5837-4f2a-bbe6-6d3dc250b83d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:03.365238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc93a0-5837-4f2a-bbe6-6d3dc250b83d service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:03.381864 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1161/2337] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:59:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:59:03.893624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f71637-60e9-4311-8bfc-2cab3c07de5e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:03.898099 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-39f71637-60e9-4311-8bfc-2cab3c07de5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:03.921200 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f71637-60e9-4311-8bfc-2cab3c07de5e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:03.926963 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a885f-2c5e-4240-9576-e5990292ca35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:03.929073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a885f-2c5e-4240-9576-e5990292ca35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:03.929073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a885f-2c5e-4240-9576-e5990292ca35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:03.929073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a885f-2c5e-4240-9576-e5990292ca35 admin admin] PATH_INFO: `/v3/projects/1650e51a164f4365a5478ae2909017c7` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:03.929883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a885f-2c5e-4240-9576-e5990292ca35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1650e51a164f4365a5478ae2909017c7)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:03.932723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a885f-2c5e-4240-9576-e5990292ca35 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:04.002388 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-081a885f-2c5e-4240-9576-e5990292ca35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1650e51a164f4365a5478ae2909017c7', 'request_id': 'req-081a885f-2c5e-4240-9576-e5990292ca35'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:04.012333 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-081a885f-2c5e-4240-9576-e5990292ca35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1650e51a164f4365a5478ae2909017c7', 'request_id': 'req-081a885f-2c5e-4240-9576-e5990292ca35'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:04.021872 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-081a885f-2c5e-4240-9576-e5990292ca35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1650e51a164f4365a5478ae2909017c7', 'request_id': 'req-081a885f-2c5e-4240-9576-e5990292ca35'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:04.041036 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-081a885f-2c5e-4240-9576-e5990292ca35 admin admin] The token cache is being invalidate because project 1650e51a164f4365a5478ae2909017c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:59:04.041403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-081a885f-2c5e-4240-9576-e5990292ca35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:04.042209 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-081a885f-2c5e-4240-9576-e5990292ca35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:04.044277 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1177/2338] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:59:03 2022] DELETE /identity/v3/projects/1650e51a164f4365a5478ae2909017c7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:04.049442 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba05d0-3bd5-44e7-918a-5571660297cf None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:04.049987 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba05d0-3bd5-44e7-918a-5571660297cf None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:04.050145 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba05d0-3bd5-44e7-918a-5571660297cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:04.070104 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-f7ba05d0-3bd5-44e7-918a-5571660297cf None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:04.072068 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f7ba05d0-3bd5-44e7-918a-5571660297cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:04.157457 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1162/2339] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:59:04.159582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151128c0-3824-46e7-9072-1c7967efd347 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:04.159873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151128c0-3824-46e7-9072-1c7967efd347 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:04.160177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151128c0-3824-46e7-9072-1c7967efd347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:04.181577 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-151128c0-3824-46e7-9072-1c7967efd347 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:04.182867 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-151128c0-3824-46e7-9072-1c7967efd347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:04.193436 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1178/2340] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:04.197364 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba05d0-3bd5-44e7-918a-5571660297cf None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:04.210328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba05d0-3bd5-44e7-918a-5571660297cf None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:04.219350 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfff83d-ab1d-4af6-b6b8-906060119fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:04.220502 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfff83d-ab1d-4af6-b6b8-906060119fb6 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:04.220997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfff83d-ab1d-4af6-b6b8-906060119fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:04.221461 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfff83d-ab1d-4af6-b6b8-906060119fb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:04.223076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfff83d-ab1d-4af6-b6b8-906060119fb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:59:04.223614 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfff83d-ab1d-4af6-b6b8-906060119fb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:04.228202 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfff83d-ab1d-4af6-b6b8-906060119fb6 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:04.240762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:04.253522 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151128c0-3824-46e7-9072-1c7967efd347 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:04.265289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1163/2341] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:59:04.267315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151128c0-3824-46e7-9072-1c7967efd347 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:04.273390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5ec9a-07ce-472a-ab03-e41ff261d11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:04.274819 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5ec9a-07ce-472a-ab03-e41ff261d11a admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:04.275148 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5ec9a-07ce-472a-ab03-e41ff261d11a admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:04.275634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5ec9a-07ce-472a-ab03-e41ff261d11a admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:04.276442 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a5ec9a-07ce-472a-ab03-e41ff261d11a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:04.278675 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a5ec9a-07ce-472a-ab03-e41ff261d11a admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:04.309692 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1179/2342] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:59:04 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:59:04.313447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfff83d-ab1d-4af6-b6b8-906060119fb6 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:04.338672 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfff83d-ab1d-4af6-b6b8-906060119fb6 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:04.437915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c0dde-3329-4168-83a2-9c570d6ae987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:04.438987 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c0dde-3329-4168-83a2-9c570d6ae987 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:04.439303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c0dde-3329-4168-83a2-9c570d6ae987 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:04.439811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c0dde-3329-4168-83a2-9c570d6ae987 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:04.440819 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c0dde-3329-4168-83a2-9c570d6ae987 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:04.443195 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c0dde-3329-4168-83a2-9c570d6ae987 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:04.467529 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1164/2343] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:59:04 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:59:04.472205 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5ec9a-07ce-472a-ab03-e41ff261d11a admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:04.488376 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5ec9a-07ce-472a-ab03-e41ff261d11a admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:04.498579 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739a3b0-75f2-4629-a6fe-9970f779f9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:04.499811 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739a3b0-75f2-4629-a6fe-9970f779f9c1 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:04.500319 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739a3b0-75f2-4629-a6fe-9970f779f9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:04.500706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739a3b0-75f2-4629-a6fe-9970f779f9c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:04.502329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739a3b0-75f2-4629-a6fe-9970f779f9c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:04.505818 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739a3b0-75f2-4629-a6fe-9970f779f9c1 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:04.510946 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:04.533006 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1180/2344] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:59:04.537151 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c0dde-3329-4168-83a2-9c570d6ae987 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:04.560548 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c0dde-3329-4168-83a2-9c570d6ae987 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:04.570199 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0beb48-a60b-4a10-921c-714df108f4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:04.571980 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0beb48-a60b-4a10-921c-714df108f4a0 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:04.572958 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0beb48-a60b-4a10-921c-714df108f4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:04.573858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0beb48-a60b-4a10-921c-714df108f4a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:04.575507 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0beb48-a60b-4a10-921c-714df108f4a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:04.580925 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0beb48-a60b-4a10-921c-714df108f4a0 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:04.608194 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1165/2345] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:59:04.611329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739a3b0-75f2-4629-a6fe-9970f779f9c1 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:04.624100 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739a3b0-75f2-4629-a6fe-9970f779f9c1 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:04.631039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df71f5-7c64-4e79-930c-c21a62d73249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:04.632870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df71f5-7c64-4e79-930c-c21a62d73249 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:04.634949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df71f5-7c64-4e79-930c-c21a62d73249 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:04.634949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df71f5-7c64-4e79-930c-c21a62d73249 admin admin] PATH_INFO: `/v3/projects/fcdfda0aa9b04628aef510eb6c5970cb/users/a5cc42ccf2534f1abced22f3dffbda42/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:04.682623 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df71f5-7c64-4e79-930c-c21a62d73249 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcdfda0aa9b04628aef510eb6c5970cb, user_id=a5cc42ccf2534f1abced22f3dffbda42, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:04.691201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df71f5-7c64-4e79-930c-c21a62d73249 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:04.758735 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1181/2346] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:59:04 2022] PUT /identity/v3/projects/fcdfda0aa9b04628aef510eb6c5970cb/users/a5cc42ccf2534f1abced22f3dffbda42/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:04.762312 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbea2e-a63b-4c0c-a1ae-8951df00ccc5 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:04.762699 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbea2e-a63b-4c0c-a1ae-8951df00ccc5 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:04.763109 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbea2e-a63b-4c0c-a1ae-8951df00ccc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:04.809795 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-61cbea2e-a63b-4c0c-a1ae-8951df00ccc5 None None] MFA Rules not processed for user `a5cc42ccf2534f1abced22f3dffbda42`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:04.811122 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-61cbea2e-a63b-4c0c-a1ae-8951df00ccc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:04.945179 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1166/2347] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:59:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:04.960681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df71f5-7c64-4e79-930c-c21a62d73249 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:04.962686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d2df71f5-7c64-4e79-930c-c21a62d73249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:05.154935 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df71f5-7c64-4e79-930c-c21a62d73249 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:05.161942 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd27ea1-d405-41b6-9be8-9657ef499a00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:05.162892 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd27ea1-d405-41b6-9be8-9657ef499a00 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:05.164862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd27ea1-d405-41b6-9be8-9657ef499a00 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:05.165795 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd27ea1-d405-41b6-9be8-9657ef499a00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:05.187745 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd27ea1-d405-41b6-9be8-9657ef499a00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:05.194499 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd27ea1-d405-41b6-9be8-9657ef499a00 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:05.234279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1182/2348] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:59:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:05.257048 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbea2e-a63b-4c0c-a1ae-8951df00ccc5 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:05.412596 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbea2e-a63b-4c0c-a1ae-8951df00ccc5 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:05.434544 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f6776-7077-40ae-b37a-9d35f609c102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:05.443254 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f6776-7077-40ae-b37a-9d35f609c102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:05.443576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f6776-7077-40ae-b37a-9d35f609c102 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:05.443969 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f6776-7077-40ae-b37a-9d35f609c102 admin admin] PATH_INFO: `/v3/users/a5cc42ccf2534f1abced22f3dffbda42` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:05.448012 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f6776-7077-40ae-b37a-9d35f609c102 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5cc42ccf2534f1abced22f3dffbda42)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:05.450043 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f6776-7077-40ae-b37a-9d35f609c102 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:05.460091 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:05.552401 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-527f6776-7077-40ae-b37a-9d35f609c102 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5cc42ccf2534f1abced22f3dffbda42'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:05.576623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-527f6776-7077-40ae-b37a-9d35f609c102 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5cc42ccf2534f1abced22f3dffbda42'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:05.591936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-527f6776-7077-40ae-b37a-9d35f609c102 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5cc42ccf2534f1abced22f3dffbda42'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:05.592872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-527f6776-7077-40ae-b37a-9d35f609c102 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5cc42ccf2534f1abced22f3dffbda42'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:05.593666 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-527f6776-7077-40ae-b37a-9d35f609c102 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5cc42ccf2534f1abced22f3dffbda42'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:05.614264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1167/2349] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:59:05 2022] DELETE /identity/v3/users/a5cc42ccf2534f1abced22f3dffbda42 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:05.620032 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2364b-0d82-492b-a257-f4e12cc3c76b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:05.620399 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2364b-0d82-492b-a257-f4e12cc3c76b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:05.621101 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2364b-0d82-492b-a257-f4e12cc3c76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:05.645989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b6b2364b-0d82-492b-a257-f4e12cc3c76b None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:05.647375 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b6b2364b-0d82-492b-a257-f4e12cc3c76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:05.776319 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1183/2350] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:59:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:05.783295 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f6776-7077-40ae-b37a-9d35f609c102 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:05.785024 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-527f6776-7077-40ae-b37a-9d35f609c102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:05.888556 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f6776-7077-40ae-b37a-9d35f609c102 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:05.893652 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81d06a-305f-4284-b491-950e415f4ac3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:05.900520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81d06a-305f-4284-b491-950e415f4ac3 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:05.900520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81d06a-305f-4284-b491-950e415f4ac3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:05.900520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81d06a-305f-4284-b491-950e415f4ac3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:05.922581 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81d06a-305f-4284-b491-950e415f4ac3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:05.924390 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81d06a-305f-4284-b491-950e415f4ac3 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:05.966159 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1168/2351] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:59:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:59:06.879042 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2364b-0d82-492b-a257-f4e12cc3c76b None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:06.886889 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b6b2364b-0d82-492b-a257-f4e12cc3c76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:06.930647 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2364b-0d82-492b-a257-f4e12cc3c76b None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:06.952009 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cec1d6-ca89-412c-a448-818f67f8ab28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:06.954277 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cec1d6-ca89-412c-a448-818f67f8ab28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:06.954722 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cec1d6-ca89-412c-a448-818f67f8ab28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:06.969249 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cec1d6-ca89-412c-a448-818f67f8ab28 admin admin] PATH_INFO: `/v3/projects/fcdfda0aa9b04628aef510eb6c5970cb` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:06.971899 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cec1d6-ca89-412c-a448-818f67f8ab28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcdfda0aa9b04628aef510eb6c5970cb)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:06.990201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cec1d6-ca89-412c-a448-818f67f8ab28 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:07.176681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-96cec1d6-ca89-412c-a448-818f67f8ab28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcdfda0aa9b04628aef510eb6c5970cb', 'request_id': 'req-96cec1d6-ca89-412c-a448-818f67f8ab28'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:07.190346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-96cec1d6-ca89-412c-a448-818f67f8ab28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcdfda0aa9b04628aef510eb6c5970cb', 'request_id': 'req-96cec1d6-ca89-412c-a448-818f67f8ab28'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:07.202787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-96cec1d6-ca89-412c-a448-818f67f8ab28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcdfda0aa9b04628aef510eb6c5970cb', 'request_id': 'req-96cec1d6-ca89-412c-a448-818f67f8ab28'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:07.218293 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:07.226096 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-96cec1d6-ca89-412c-a448-818f67f8ab28 admin admin] The token cache is being invalidate because project fcdfda0aa9b04628aef510eb6c5970cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:59:07.227498 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-96cec1d6-ca89-412c-a448-818f67f8ab28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:07.229473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-96cec1d6-ca89-412c-a448-818f67f8ab28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:07.233087 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1184/2352] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:59:06 2022] DELETE /identity/v3/projects/fcdfda0aa9b04628aef510eb6c5970cb => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:07.238498 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d521135-2803-410a-bf5f-49f5c51b0d41 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:07.239060 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d521135-2803-410a-bf5f-49f5c51b0d41 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:07.239292 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d521135-2803-410a-bf5f-49f5c51b0d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:07.280047 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-6d521135-2803-410a-bf5f-49f5c51b0d41 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:07.282630 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-6d521135-2803-410a-bf5f-49f5c51b0d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:07.398144 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1169/2353] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:59:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:07.406150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90c628-cc05-45ab-9e28-9e2fe3a6a116 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:07.410628 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90c628-cc05-45ab-9e28-9e2fe3a6a116 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:07.411535 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90c628-cc05-45ab-9e28-9e2fe3a6a116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:07.450436 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-fc90c628-cc05-45ab-9e28-9e2fe3a6a116 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:07.451827 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-fc90c628-cc05-45ab-9e28-9e2fe3a6a116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:07.474050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d521135-2803-410a-bf5f-49f5c51b0d41 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:07.478352 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1185/2354] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:59:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:07.529041 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d521135-2803-410a-bf5f-49f5c51b0d41 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:07.544575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896188e5-2a03-4811-a945-dd319ec54c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:07.545634 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896188e5-2a03-4811-a945-dd319ec54c56 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:07.546117 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896188e5-2a03-4811-a945-dd319ec54c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:07.546569 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896188e5-2a03-4811-a945-dd319ec54c56 admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:07.547516 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896188e5-2a03-4811-a945-dd319ec54c56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:59:07.547997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896188e5-2a03-4811-a945-dd319ec54c56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:07.550782 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896188e5-2a03-4811-a945-dd319ec54c56 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:07.607638 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1170/2355] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 14:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:59:07.620587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90c628-cc05-45ab-9e28-9e2fe3a6a116 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:07.673942 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90c628-cc05-45ab-9e28-9e2fe3a6a116 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:07.697813 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236e013-e9aa-4184-bb22-3bed3428ffb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:07.698979 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236e013-e9aa-4184-bb22-3bed3428ffb3 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:07.699833 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236e013-e9aa-4184-bb22-3bed3428ffb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:07.699833 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236e013-e9aa-4184-bb22-3bed3428ffb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:07.701017 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a236e013-e9aa-4184-bb22-3bed3428ffb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:07.709613 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a236e013-e9aa-4184-bb22-3bed3428ffb3 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:07.791386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1186/2356] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:59:07 2022] POST /identity/v3/projects => generated 383 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:59:07.796146 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896188e5-2a03-4811-a945-dd319ec54c56 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:07.822355 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896188e5-2a03-4811-a945-dd319ec54c56 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:07.880379 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:08.005257 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c6bbb8-b6d3-4a15-983e-6045062cbc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:08.005952 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6bbb8-b6d3-4a15-983e-6045062cbc9a admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:08.006298 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6bbb8-b6d3-4a15-983e-6045062cbc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:08.006579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6bbb8-b6d3-4a15-983e-6045062cbc9a admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:08.007359 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c6bbb8-b6d3-4a15-983e-6045062cbc9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:08.009972 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c6bbb8-b6d3-4a15-983e-6045062cbc9a admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:08.089305 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1171/2357] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:59:07 2022] POST /identity/v3/users => generated 292 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:59:08.090766 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236e013-e9aa-4184-bb22-3bed3428ffb3 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:08.112576 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236e013-e9aa-4184-bb22-3bed3428ffb3 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:08.126839 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10561f94-cf43-4519-b85a-c358b9f079ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:08.127696 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10561f94-cf43-4519-b85a-c358b9f079ab admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:08.130561 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10561f94-cf43-4519-b85a-c358b9f079ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:08.131318 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10561f94-cf43-4519-b85a-c358b9f079ab admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:08.132394 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10561f94-cf43-4519-b85a-c358b9f079ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:08.146605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10561f94-cf43-4519-b85a-c358b9f079ab admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:08.167225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1187/2358] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:59:08.172248 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c6bbb8-b6d3-4a15-983e-6045062cbc9a admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:08.191555 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c6bbb8-b6d3-4a15-983e-6045062cbc9a admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:08.205430 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af031ec-18bd-41aa-bd07-947ee8920a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:08.206126 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af031ec-18bd-41aa-bd07-947ee8920a58 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:08.209758 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af031ec-18bd-41aa-bd07-947ee8920a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:08.210163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af031ec-18bd-41aa-bd07-947ee8920a58 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:08.211851 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af031ec-18bd-41aa-bd07-947ee8920a58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:08.219569 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af031ec-18bd-41aa-bd07-947ee8920a58 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:08.246406 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1172/2359] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:59:08.248707 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10561f94-cf43-4519-b85a-c358b9f079ab admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:08.266328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10561f94-cf43-4519-b85a-c358b9f079ab admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:08.302393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae09ec-a286-455a-bae3-be4cb0677c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:08.303779 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae09ec-a286-455a-bae3-be4cb0677c4b admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:08.304177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae09ec-a286-455a-bae3-be4cb0677c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:08.304650 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae09ec-a286-455a-bae3-be4cb0677c4b admin admin] PATH_INFO: `/v3/projects/b0ba5fe9d30041388e3967d28d806943/users/9715e38cc81e4bc78c41a3a92716406b/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:08.348549 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ae09ec-a286-455a-bae3-be4cb0677c4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0ba5fe9d30041388e3967d28d806943, user_id=9715e38cc81e4bc78c41a3a92716406b, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:08.353577 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ae09ec-a286-455a-bae3-be4cb0677c4b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:08.394873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1188/2360] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:59:08 2022] PUT /identity/v3/projects/b0ba5fe9d30041388e3967d28d806943/users/9715e38cc81e4bc78c41a3a92716406b/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:08.397163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992bbf3-0435-46de-86de-b2c16b817311 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:08.397621 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992bbf3-0435-46de-86de-b2c16b817311 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:08.397922 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992bbf3-0435-46de-86de-b2c16b817311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:08.457145 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-f992bbf3-0435-46de-86de-b2c16b817311 None None] MFA Rules not processed for user `9715e38cc81e4bc78c41a3a92716406b`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:08.458666 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f992bbf3-0435-46de-86de-b2c16b817311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:08.601307 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1173/2361] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:59:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:08.603180 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae09ec-a286-455a-bae3-be4cb0677c4b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:08.815601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae09ec-a286-455a-bae3-be4cb0677c4b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:08.823326 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ceb3f-8199-4167-89d1-5ec9344184fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:08.824256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ceb3f-8199-4167-89d1-5ec9344184fd admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:08.824752 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ceb3f-8199-4167-89d1-5ec9344184fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:08.825223 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ceb3f-8199-4167-89d1-5ec9344184fd admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:08.826179 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ceb3f-8199-4167-89d1-5ec9344184fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:08.828499 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ceb3f-8199-4167-89d1-5ec9344184fd admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:08.889940 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1189/2362] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:59:08 2022] POST /identity/v3/projects => generated 383 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:59:08.899111 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992bbf3-0435-46de-86de-b2c16b817311 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:08.926203 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992bbf3-0435-46de-86de-b2c16b817311 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:09.094675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e216b-bfac-4364-8079-66c16fb6e3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:09.107125 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e216b-bfac-4364-8079-66c16fb6e3ae admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:09.107528 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e216b-bfac-4364-8079-66c16fb6e3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:09.107958 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e216b-bfac-4364-8079-66c16fb6e3ae admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:09.108912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e216b-bfac-4364-8079-66c16fb6e3ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:09.111781 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e216b-bfac-4364-8079-66c16fb6e3ae admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:09.208118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ceb3f-8199-4167-89d1-5ec9344184fd admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:09.209076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1174/2363] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:59:08 2022] POST /identity/v3/users => generated 292 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:59:09.230109 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ceb3f-8199-4167-89d1-5ec9344184fd admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:09.259286 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ac41c-1e66-4994-893e-620e66b81846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:09.260391 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ac41c-1e66-4994-893e-620e66b81846 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:09.261790 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ac41c-1e66-4994-893e-620e66b81846 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:09.262402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ac41c-1e66-4994-893e-620e66b81846 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:09.265133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ac41c-1e66-4994-893e-620e66b81846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:09.268904 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ac41c-1e66-4994-893e-620e66b81846 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:09.317988 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1190/2364] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:59:09.318237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e216b-bfac-4364-8079-66c16fb6e3ae admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:09.341973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e216b-bfac-4364-8079-66c16fb6e3ae admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:09.358156 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61992e88-cd2d-45da-9101-d723cb074755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:09.358971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61992e88-cd2d-45da-9101-d723cb074755 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:09.359303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61992e88-cd2d-45da-9101-d723cb074755 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:09.361888 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61992e88-cd2d-45da-9101-d723cb074755 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:09.362890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61992e88-cd2d-45da-9101-d723cb074755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:09.374024 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61992e88-cd2d-45da-9101-d723cb074755 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:09.409401 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ac41c-1e66-4994-893e-620e66b81846 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:09.417472 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1175/2365] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:59:09.422652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ac41c-1e66-4994-893e-620e66b81846 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:09.430515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0bff3-e0ff-4744-91c5-4dc54dbbb77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:09.431767 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0bff3-e0ff-4744-91c5-4dc54dbbb77c admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:09.432191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0bff3-e0ff-4744-91c5-4dc54dbbb77c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:09.432716 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0bff3-e0ff-4744-91c5-4dc54dbbb77c admin admin] PATH_INFO: `/v3/projects/0a02b1cd031144eba63516a948078261/users/269901677f22466f84cf49cd5be06eec/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:09.478173 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0bff3-e0ff-4744-91c5-4dc54dbbb77c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a02b1cd031144eba63516a948078261, user_id=269901677f22466f84cf49cd5be06eec, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:09.488786 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0bff3-e0ff-4744-91c5-4dc54dbbb77c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:09.515877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1191/2366] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:59:09 2022] PUT /identity/v3/projects/0a02b1cd031144eba63516a948078261/users/269901677f22466f84cf49cd5be06eec/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:09.528898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911fe8e-f5e0-4a3a-9685-40401b42a37e None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:09.529236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911fe8e-f5e0-4a3a-9685-40401b42a37e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:09.529709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911fe8e-f5e0-4a3a-9685-40401b42a37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:09.536593 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:09.614885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-8911fe8e-f5e0-4a3a-9685-40401b42a37e None None] MFA Rules not processed for user `269901677f22466f84cf49cd5be06eec`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:09.615725 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-8911fe8e-f5e0-4a3a-9685-40401b42a37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:09.724493 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1176/2367] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:59:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:09.729463 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0bff3-e0ff-4744-91c5-4dc54dbbb77c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:09.813339 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:09.838554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0bff3-e0ff-4744-91c5-4dc54dbbb77c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:09.849788 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61037dee-3dfd-4f83-8de2-84f684489755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:09.852278 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61037dee-3dfd-4f83-8de2-84f684489755 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:09.853177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61037dee-3dfd-4f83-8de2-84f684489755 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:09.853650 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61037dee-3dfd-4f83-8de2-84f684489755 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:09.855322 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61037dee-3dfd-4f83-8de2-84f684489755 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:09.857575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61037dee-3dfd-4f83-8de2-84f684489755 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:09.902785 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1192/2368] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 14:59:09 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:59:09.907038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911fe8e-f5e0-4a3a-9685-40401b42a37e None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:09.936517 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911fe8e-f5e0-4a3a-9685-40401b42a37e None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:10.085866 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c01030-c695-441e-8c0f-e4bdfe06ab50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:10.086631 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c01030-c695-441e-8c0f-e4bdfe06ab50 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:10.086908 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c01030-c695-441e-8c0f-e4bdfe06ab50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:10.087179 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c01030-c695-441e-8c0f-e4bdfe06ab50 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:10.087896 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c01030-c695-441e-8c0f-e4bdfe06ab50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:10.089926 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c01030-c695-441e-8c0f-e4bdfe06ab50 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:10.124253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1177/2369] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 14:59:09 2022] POST /identity/v3/users => generated 299 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:59:10.129139 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61037dee-3dfd-4f83-8de2-84f684489755 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:10.144988 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61037dee-3dfd-4f83-8de2-84f684489755 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:10.152171 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf976e-fa60-4298-8e0d-dd04b020943e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:10.153414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf976e-fa60-4298-8e0d-dd04b020943e admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:10.153852 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf976e-fa60-4298-8e0d-dd04b020943e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:10.154255 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf976e-fa60-4298-8e0d-dd04b020943e admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:10.155256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf976e-fa60-4298-8e0d-dd04b020943e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:10.157248 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf976e-fa60-4298-8e0d-dd04b020943e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:10.171386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1193/2370] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:59:10.176182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c01030-c695-441e-8c0f-e4bdfe06ab50 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:10.200691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c01030-c695-441e-8c0f-e4bdfe06ab50 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:10.213376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc596260-bd8d-494f-b055-b9e6b75215d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:10.213376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc596260-bd8d-494f-b055-b9e6b75215d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:10.213376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc596260-bd8d-494f-b055-b9e6b75215d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:10.213376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc596260-bd8d-494f-b055-b9e6b75215d4 admin admin] PATH_INFO: `/v3/projects/803d0eb4f2d04a87ae4d5b16bcf2aef9/users/9fe32e62a7634397b31796c88bad41e9/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:10.241049 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc596260-bd8d-494f-b055-b9e6b75215d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=803d0eb4f2d04a87ae4d5b16bcf2aef9, user_id=9fe32e62a7634397b31796c88bad41e9, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:10.245495 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc596260-bd8d-494f-b055-b9e6b75215d4 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:10.266502 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1178/2371] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:59:10 2022] PUT /identity/v3/projects/803d0eb4f2d04a87ae4d5b16bcf2aef9/users/9fe32e62a7634397b31796c88bad41e9/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:10.270995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf976e-fa60-4298-8e0d-dd04b020943e admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:10.290930 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf976e-fa60-4298-8e0d-dd04b020943e admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:10.332913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc596260-bd8d-494f-b055-b9e6b75215d4 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:10.340861 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-fc596260-bd8d-494f-b055-b9e6b75215d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:10.466872 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313119b1-9710-483f-a2a1-2bb1a9b3bb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:10.467791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313119b1-9710-483f-a2a1-2bb1a9b3bb8d admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:10.467997 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313119b1-9710-483f-a2a1-2bb1a9b3bb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:10.468320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313119b1-9710-483f-a2a1-2bb1a9b3bb8d admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:10.469213 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313119b1-9710-483f-a2a1-2bb1a9b3bb8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:10.474743 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313119b1-9710-483f-a2a1-2bb1a9b3bb8d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:10.492127 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:10.493696 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1194/2372] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 14:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:59:10.498632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313119b1-9710-483f-a2a1-2bb1a9b3bb8d admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:10.502137 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc596260-bd8d-494f-b055-b9e6b75215d4 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:10.511805 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059b2dd-e89c-4516-aefa-077e080ef3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:10.512876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059b2dd-e89c-4516-aefa-077e080ef3dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:10.513022 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313119b1-9710-483f-a2a1-2bb1a9b3bb8d admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:10.513408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059b2dd-e89c-4516-aefa-077e080ef3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:10.514172 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059b2dd-e89c-4516-aefa-077e080ef3dd admin admin] PATH_INFO: `/v3/users/26a977b132994a42a0c830224ca1c1f8` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:10.516228 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059b2dd-e89c-4516-aefa-077e080ef3dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=26a977b132994a42a0c830224ca1c1f8)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:10.519242 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059b2dd-e89c-4516-aefa-077e080ef3dd admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:10.521217 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4152c-c977-4885-a01c-89c3f8b52d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:10.523048 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4152c-c977-4885-a01c-89c3f8b52d4d admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:10.523764 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4152c-c977-4885-a01c-89c3f8b52d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:10.524134 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4152c-c977-4885-a01c-89c3f8b52d4d admin admin] PATH_INFO: `/v3/projects/803d0eb4f2d04a87ae4d5b16bcf2aef9/users/9fe32e62a7634397b31796c88bad41e9/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:10.529110 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee4152c-c977-4885-a01c-89c3f8b52d4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=803d0eb4f2d04a87ae4d5b16bcf2aef9, user_id=9fe32e62a7634397b31796c88bad41e9, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:10.546953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee4152c-c977-4885-a01c-89c3f8b52d4d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:10.606183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1195/2373] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 14:59:10 2022] PUT /identity/v3/projects/803d0eb4f2d04a87ae4d5b16bcf2aef9/users/9fe32e62a7634397b31796c88bad41e9/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:59:10.614831 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf81692-881a-417a-b52f-2bc2ee16ef49 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:10.615127 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf81692-881a-417a-b52f-2bc2ee16ef49 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:10.615474 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf81692-881a-417a-b52f-2bc2ee16ef49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:10.697680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-9059b2dd-e89c-4516-aefa-077e080ef3dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26a977b132994a42a0c830224ca1c1f8'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:10.718668 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-1cf81692-881a-417a-b52f-2bc2ee16ef49 None None] MFA Rules not processed for user `3712ad1188fe405b9b25d3f1254d6746`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:10.722281 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1cf81692-881a-417a-b52f-2bc2ee16ef49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:10.726360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-9059b2dd-e89c-4516-aefa-077e080ef3dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26a977b132994a42a0c830224ca1c1f8'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:10.740414 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-9059b2dd-e89c-4516-aefa-077e080ef3dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26a977b132994a42a0c830224ca1c1f8'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:10.741334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-9059b2dd-e89c-4516-aefa-077e080ef3dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26a977b132994a42a0c830224ca1c1f8'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:10.743003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-9059b2dd-e89c-4516-aefa-077e080ef3dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26a977b132994a42a0c830224ca1c1f8'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:10.778685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1179/2374] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:59:10 2022] DELETE /identity/v3/users/26a977b132994a42a0c830224ca1c1f8 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:10.781380 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb11eae-8383-4165-87ca-930c695482a8 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:10.781558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb11eae-8383-4165-87ca-930c695482a8 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:10.781793 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb11eae-8383-4165-87ca-930c695482a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:10.830607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-0eb11eae-8383-4165-87ca-930c695482a8 None None] MFA Rules not processed for user `9fe32e62a7634397b31796c88bad41e9`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:10.836231 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-0eb11eae-8383-4165-87ca-930c695482a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:10.915306 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1196/2375] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:59:10 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:10.915306 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59f053-3b03-4ba3-8e80-e297e44c4395 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:10.915306 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59f053-3b03-4ba3-8e80-e297e44c4395 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:10.915306 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59f053-3b03-4ba3-8e80-e297e44c4395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:10.959505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-ec59f053-3b03-4ba3-8e80-e297e44c4395 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:10.964733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ec59f053-3b03-4ba3-8e80-e297e44c4395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:11.025143 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1180/2376] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:59:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:11.042685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb11eae-8383-4165-87ca-930c695482a8 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:11.045403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:11.047925 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-0eb11eae-8383-4165-87ca-930c695482a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:11.133836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1197/2377] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:11.134651 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59f053-3b03-4ba3-8e80-e297e44c4395 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:11.136066 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ec59f053-3b03-4ba3-8e80-e297e44c4395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:11.168786 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb11eae-8383-4165-87ca-930c695482a8 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:11.174745 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc153f7-eed2-4f9f-9e28-bcd2a934520d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:11.175462 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc153f7-eed2-4f9f-9e28-bcd2a934520d service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:11.175792 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc153f7-eed2-4f9f-9e28-bcd2a934520d service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:11.176070 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc153f7-eed2-4f9f-9e28-bcd2a934520d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:11.180807 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-9dc153f7-eed2-4f9f-9e28-bcd2a934520d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:11.205500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:11.289728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc153f7-eed2-4f9f-9e28-bcd2a934520d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:11.291672 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc153f7-eed2-4f9f-9e28-bcd2a934520d service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:11.300022 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59f053-3b03-4ba3-8e80-e297e44c4395 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:11.306594 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1181/2378] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:59:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:11.309882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc153f7-eed2-4f9f-9e28-bcd2a934520d service cinder] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:11.310452 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146bab5-0965-48a9-ba48-6cbeb0da8495 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:11.311341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146bab5-0965-48a9-ba48-6cbeb0da8495 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:11.312014 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146bab5-0965-48a9-ba48-6cbeb0da8495 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:11.312382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-9dc153f7-eed2-4f9f-9e28-bcd2a934520d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:11.312619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146bab5-0965-48a9-ba48-6cbeb0da8495 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:11.314816 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9146bab5-0965-48a9-ba48-6cbeb0da8495 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:11.402498 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc153f7-eed2-4f9f-9e28-bcd2a934520d service cinder] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:11.407040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847ec5f-4a49-4758-bc83-2aab6f3b57cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:11.408040 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847ec5f-4a49-4758-bc83-2aab6f3b57cb service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:11.408452 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847ec5f-4a49-4758-bc83-2aab6f3b57cb service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:11.408739 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847ec5f-4a49-4758-bc83-2aab6f3b57cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:11.419619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146bab5-0965-48a9-ba48-6cbeb0da8495 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:11.422238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847ec5f-4a49-4758-bc83-2aab6f3b57cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:11.423030 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146bab5-0965-48a9-ba48-6cbeb0da8495 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:11.423591 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847ec5f-4a49-4758-bc83-2aab6f3b57cb service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:11.443947 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1182/2380] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:59:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:11.465975 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1198/2379] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:59:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:11.963314 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847ec5f-4a49-4758-bc83-2aab6f3b57cb service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:11.968089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-5847ec5f-4a49-4758-bc83-2aab6f3b57cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:11.999516 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847ec5f-4a49-4758-bc83-2aab6f3b57cb service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:12.019946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b391c-3d94-415d-89ad-a86ec539a984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:12.021499 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b391c-3d94-415d-89ad-a86ec539a984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:12.025723 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b391c-3d94-415d-89ad-a86ec539a984 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:12.026556 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b391c-3d94-415d-89ad-a86ec539a984 admin admin] PATH_INFO: `/v3/projects/147f6f739c6e4e1bab0b20768f3131c2` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:12.028209 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b391c-3d94-415d-89ad-a86ec539a984 admin admin] RBAC: Authorizing `identity:delete_project(project_id=147f6f739c6e4e1bab0b20768f3131c2)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:12.030574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b391c-3d94-415d-89ad-a86ec539a984 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:12.110441 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c34b391c-3d94-415d-89ad-a86ec539a984 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '147f6f739c6e4e1bab0b20768f3131c2', 'request_id': 'req-c34b391c-3d94-415d-89ad-a86ec539a984'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:12.120611 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c34b391c-3d94-415d-89ad-a86ec539a984 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '147f6f739c6e4e1bab0b20768f3131c2', 'request_id': 'req-c34b391c-3d94-415d-89ad-a86ec539a984'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:12.132145 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c34b391c-3d94-415d-89ad-a86ec539a984 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '147f6f739c6e4e1bab0b20768f3131c2', 'request_id': 'req-c34b391c-3d94-415d-89ad-a86ec539a984'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:12.157574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c34b391c-3d94-415d-89ad-a86ec539a984 admin admin] The token cache is being invalidate because project 147f6f739c6e4e1bab0b20768f3131c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:59:12.157935 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c34b391c-3d94-415d-89ad-a86ec539a984 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:12.158603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c34b391c-3d94-415d-89ad-a86ec539a984 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:12.161185 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1183/2381] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:59:11 2022] DELETE /identity/v3/projects/147f6f739c6e4e1bab0b20768f3131c2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:13.119052 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146bab5-0965-48a9-ba48-6cbeb0da8495 service nova] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:13.133093 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146bab5-0965-48a9-ba48-6cbeb0da8495 service nova] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:13.196380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636a90e-3a42-4043-b7d3-63aafc3623fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:13.197188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636a90e-3a42-4043-b7d3-63aafc3623fa service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:13.197757 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636a90e-3a42-4043-b7d3-63aafc3623fa service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:13.198175 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636a90e-3a42-4043-b7d3-63aafc3623fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:13.206148 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1636a90e-3a42-4043-b7d3-63aafc3623fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:13.236041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:13.316025 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636a90e-3a42-4043-b7d3-63aafc3623fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:13.318160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636a90e-3a42-4043-b7d3-63aafc3623fa service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:13.330923 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1199/2382] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:59:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:14.293443 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b391c-3d94-415d-89ad-a86ec539a984 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:14.295056 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-c34b391c-3d94-415d-89ad-a86ec539a984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:14.308539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b391c-3d94-415d-89ad-a86ec539a984 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:14.312385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390b79f-9383-4a52-ab7f-bf9b1800b5eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:14.313148 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390b79f-9383-4a52-ab7f-bf9b1800b5eb service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:14.313491 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390b79f-9383-4a52-ab7f-bf9b1800b5eb service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:14.313892 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390b79f-9383-4a52-ab7f-bf9b1800b5eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:14.315859 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d390b79f-9383-4a52-ab7f-bf9b1800b5eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:14.365929 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:14.401544 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390b79f-9383-4a52-ab7f-bf9b1800b5eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:14.403696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390b79f-9383-4a52-ab7f-bf9b1800b5eb service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:14.418279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1184/2383] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:59:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:18.568167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2faaf3-728b-4f66-b383-8489ffae2041 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:18.569194 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2faaf3-728b-4f66-b383-8489ffae2041 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:18.569484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2faaf3-728b-4f66-b383-8489ffae2041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:18.593461 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-fc2faaf3-728b-4f66-b383-8489ffae2041 None None] MFA Rules not processed for user `e60a5e751f22450c8e366c85b37af1f6`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:18.595057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-fc2faaf3-728b-4f66-b383-8489ffae2041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:18.605521 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390b79f-9383-4a52-ab7f-bf9b1800b5eb service cinder] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:18.609709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d390b79f-9383-4a52-ab7f-bf9b1800b5eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:18.690394 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1200/2384] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:59:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:18.696992 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390b79f-9383-4a52-ab7f-bf9b1800b5eb service cinder] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:18.704582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598bc1f6-f3bd-44d9-a51a-9c86e0621870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:18.705470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bc1f6-f3bd-44d9-a51a-9c86e0621870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:18.705750 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bc1f6-f3bd-44d9-a51a-9c86e0621870 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:18.706043 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bc1f6-f3bd-44d9-a51a-9c86e0621870 admin admin] PATH_INFO: `/v3/users/9715e38cc81e4bc78c41a3a92716406b` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:18.707265 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598bc1f6-f3bd-44d9-a51a-9c86e0621870 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9715e38cc81e4bc78c41a3a92716406b)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:18.709419 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598bc1f6-f3bd-44d9-a51a-9c86e0621870 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:18.760337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-598bc1f6-f3bd-44d9-a51a-9c86e0621870 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9715e38cc81e4bc78c41a3a92716406b'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:18.772936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-598bc1f6-f3bd-44d9-a51a-9c86e0621870 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9715e38cc81e4bc78c41a3a92716406b'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:18.780610 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-598bc1f6-f3bd-44d9-a51a-9c86e0621870 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9715e38cc81e4bc78c41a3a92716406b'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:18.781395 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-598bc1f6-f3bd-44d9-a51a-9c86e0621870 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9715e38cc81e4bc78c41a3a92716406b'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:18.782042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-598bc1f6-f3bd-44d9-a51a-9c86e0621870 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9715e38cc81e4bc78c41a3a92716406b'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:18.796917 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1185/2385] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:59:18 2022] DELETE /identity/v3/users/9715e38cc81e4bc78c41a3a92716406b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:59:18.801603 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867a517-fec3-4e8c-9cf0-9263ca3b7790 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:18.801928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867a517-fec3-4e8c-9cf0-9263ca3b7790 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:18.802238 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867a517-fec3-4e8c-9cf0-9263ca3b7790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:18.820589 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-3867a517-fec3-4e8c-9cf0-9263ca3b7790 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:18.821802 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3867a517-fec3-4e8c-9cf0-9263ca3b7790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:18.887680 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1201/2386] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:59:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:18.899365 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598bc1f6-f3bd-44d9-a51a-9c86e0621870 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:18.901951 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-598bc1f6-f3bd-44d9-a51a-9c86e0621870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:18.984084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598bc1f6-f3bd-44d9-a51a-9c86e0621870 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:18.987935 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0a920-9b6f-4f5b-8a9f-0147b7cb28d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:18.988595 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0a920-9b6f-4f5b-8a9f-0147b7cb28d3 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:18.988872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0a920-9b6f-4f5b-8a9f-0147b7cb28d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:18.989149 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0a920-9b6f-4f5b-8a9f-0147b7cb28d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:19.000291 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b0a920-9b6f-4f5b-8a9f-0147b7cb28d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:19.001750 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b0a920-9b6f-4f5b-8a9f-0147b7cb28d3 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:19.006132 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:19.018536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1186/2387] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:59:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:19.399095 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867a517-fec3-4e8c-9cf0-9263ca3b7790 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:19.400709 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3867a517-fec3-4e8c-9cf0-9263ca3b7790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:19.416637 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867a517-fec3-4e8c-9cf0-9263ca3b7790 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:19.421676 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40693e58-3e49-4419-bc74-45e15813315b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:19.422581 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40693e58-3e49-4419-bc74-45e15813315b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:19.422887 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40693e58-3e49-4419-bc74-45e15813315b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:19.423169 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40693e58-3e49-4419-bc74-45e15813315b admin admin] PATH_INFO: `/v3/projects/b0ba5fe9d30041388e3967d28d806943` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:19.424421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40693e58-3e49-4419-bc74-45e15813315b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0ba5fe9d30041388e3967d28d806943)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:19.426434 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40693e58-3e49-4419-bc74-45e15813315b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:19.483250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-40693e58-3e49-4419-bc74-45e15813315b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0ba5fe9d30041388e3967d28d806943', 'request_id': 'req-40693e58-3e49-4419-bc74-45e15813315b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:19.490553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-40693e58-3e49-4419-bc74-45e15813315b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0ba5fe9d30041388e3967d28d806943', 'request_id': 'req-40693e58-3e49-4419-bc74-45e15813315b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:19.494160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:19.502107 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-40693e58-3e49-4419-bc74-45e15813315b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0ba5fe9d30041388e3967d28d806943', 'request_id': 'req-40693e58-3e49-4419-bc74-45e15813315b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:19.516660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-40693e58-3e49-4419-bc74-45e15813315b admin admin] The token cache is being invalidate because project b0ba5fe9d30041388e3967d28d806943 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:59:19.516951 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-40693e58-3e49-4419-bc74-45e15813315b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:19.517593 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-40693e58-3e49-4419-bc74-45e15813315b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:19.519221 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1202/2388] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:59:19 2022] DELETE /identity/v3/projects/b0ba5fe9d30041388e3967d28d806943 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:19.523524 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0a920-9b6f-4f5b-8a9f-0147b7cb28d3 service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:19.534693 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0a920-9b6f-4f5b-8a9f-0147b7cb28d3 service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:19.595847 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385ce9a-a42e-4e7e-b317-21f0fbf62aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:19.597045 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385ce9a-a42e-4e7e-b317-21f0fbf62aed admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:19.597045 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385ce9a-a42e-4e7e-b317-21f0fbf62aed admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:19.597267 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385ce9a-a42e-4e7e-b317-21f0fbf62aed admin admin] PATH_INFO: `/v3/users/269901677f22466f84cf49cd5be06eec` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:19.598505 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385ce9a-a42e-4e7e-b317-21f0fbf62aed admin admin] RBAC: Authorizing `identity:delete_user(user_id=269901677f22466f84cf49cd5be06eec)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:19.600445 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385ce9a-a42e-4e7e-b317-21f0fbf62aed admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:19.675072 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f385ce9a-a42e-4e7e-b317-21f0fbf62aed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '269901677f22466f84cf49cd5be06eec'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:19.688166 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f385ce9a-a42e-4e7e-b317-21f0fbf62aed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '269901677f22466f84cf49cd5be06eec'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:19.696437 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f385ce9a-a42e-4e7e-b317-21f0fbf62aed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '269901677f22466f84cf49cd5be06eec'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:19.697193 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f385ce9a-a42e-4e7e-b317-21f0fbf62aed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '269901677f22466f84cf49cd5be06eec'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:19.697810 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f385ce9a-a42e-4e7e-b317-21f0fbf62aed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '269901677f22466f84cf49cd5be06eec'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:19.711785 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1187/2389] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:59:19 2022] DELETE /identity/v3/users/269901677f22466f84cf49cd5be06eec => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:20.103160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40693e58-3e49-4419-bc74-45e15813315b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:20.105733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-40693e58-3e49-4419-bc74-45e15813315b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:20.131883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40693e58-3e49-4419-bc74-45e15813315b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:20.225586 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e89a13a-d2ab-4442-ab8c-a64d7ab9c1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:20.231309 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89a13a-d2ab-4442-ab8c-a64d7ab9c1ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:20.232156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89a13a-d2ab-4442-ab8c-a64d7ab9c1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:20.232661 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89a13a-d2ab-4442-ab8c-a64d7ab9c1ba admin admin] PATH_INFO: `/v3/projects/0a02b1cd031144eba63516a948078261` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:20.234580 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e89a13a-d2ab-4442-ab8c-a64d7ab9c1ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a02b1cd031144eba63516a948078261)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:20.237220 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e89a13a-d2ab-4442-ab8c-a64d7ab9c1ba admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:20.317374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1e89a13a-d2ab-4442-ab8c-a64d7ab9c1ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a02b1cd031144eba63516a948078261', 'request_id': 'req-1e89a13a-d2ab-4442-ab8c-a64d7ab9c1ba'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:20.327527 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1e89a13a-d2ab-4442-ab8c-a64d7ab9c1ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a02b1cd031144eba63516a948078261', 'request_id': 'req-1e89a13a-d2ab-4442-ab8c-a64d7ab9c1ba'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:20.334397 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1e89a13a-d2ab-4442-ab8c-a64d7ab9c1ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a02b1cd031144eba63516a948078261', 'request_id': 'req-1e89a13a-d2ab-4442-ab8c-a64d7ab9c1ba'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:20.353065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1e89a13a-d2ab-4442-ab8c-a64d7ab9c1ba admin admin] The token cache is being invalidate because project 0a02b1cd031144eba63516a948078261 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:59:20.353356 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1e89a13a-d2ab-4442-ab8c-a64d7ab9c1ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:20.354147 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1e89a13a-d2ab-4442-ab8c-a64d7ab9c1ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:20.356135 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1203/2390] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:59:20 2022] DELETE /identity/v3/projects/0a02b1cd031144eba63516a948078261 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:20.361602 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385ce9a-a42e-4e7e-b317-21f0fbf62aed admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:20.363726 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f385ce9a-a42e-4e7e-b317-21f0fbf62aed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:20.385570 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385ce9a-a42e-4e7e-b317-21f0fbf62aed admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:20.478573 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d4c88-92c3-4559-b928-338cab7db8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:20.479620 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d4c88-92c3-4559-b928-338cab7db8b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:20.479895 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d4c88-92c3-4559-b928-338cab7db8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:20.480166 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d4c88-92c3-4559-b928-338cab7db8b4 admin admin] PATH_INFO: `/v3/users/9fe32e62a7634397b31796c88bad41e9` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:20.481420 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d4c88-92c3-4559-b928-338cab7db8b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fe32e62a7634397b31796c88bad41e9)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:20.483411 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d4c88-92c3-4559-b928-338cab7db8b4 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:20.509142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:20.551428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ce7d4c88-92c3-4559-b928-338cab7db8b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fe32e62a7634397b31796c88bad41e9'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:20.567684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ce7d4c88-92c3-4559-b928-338cab7db8b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fe32e62a7634397b31796c88bad41e9'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:20.577422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ce7d4c88-92c3-4559-b928-338cab7db8b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fe32e62a7634397b31796c88bad41e9'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:20.578342 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ce7d4c88-92c3-4559-b928-338cab7db8b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fe32e62a7634397b31796c88bad41e9'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:20.579251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ce7d4c88-92c3-4559-b928-338cab7db8b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fe32e62a7634397b31796c88bad41e9'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:20.593913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1188/2391] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:59:20 2022] DELETE /identity/v3/users/9fe32e62a7634397b31796c88bad41e9 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:20.962273 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e89a13a-d2ab-4442-ab8c-a64d7ab9c1ba admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:20.964044 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1e89a13a-d2ab-4442-ab8c-a64d7ab9c1ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:20.980265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e89a13a-d2ab-4442-ab8c-a64d7ab9c1ba admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:21.053378 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f65cac-983e-486e-af27-76a2cc623202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:21.054234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f65cac-983e-486e-af27-76a2cc623202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:21.054523 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f65cac-983e-486e-af27-76a2cc623202 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:21.054807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f65cac-983e-486e-af27-76a2cc623202 admin admin] PATH_INFO: `/v3/projects/803d0eb4f2d04a87ae4d5b16bcf2aef9` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:21.056876 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f65cac-983e-486e-af27-76a2cc623202 admin admin] RBAC: Authorizing `identity:delete_project(project_id=803d0eb4f2d04a87ae4d5b16bcf2aef9)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:21.058905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f65cac-983e-486e-af27-76a2cc623202 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:21.086858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:21.114972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e4f65cac-983e-486e-af27-76a2cc623202 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '803d0eb4f2d04a87ae4d5b16bcf2aef9', 'request_id': 'req-e4f65cac-983e-486e-af27-76a2cc623202'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:21.121492 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e4f65cac-983e-486e-af27-76a2cc623202 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '803d0eb4f2d04a87ae4d5b16bcf2aef9', 'request_id': 'req-e4f65cac-983e-486e-af27-76a2cc623202'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:21.129453 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e4f65cac-983e-486e-af27-76a2cc623202 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '803d0eb4f2d04a87ae4d5b16bcf2aef9', 'request_id': 'req-e4f65cac-983e-486e-af27-76a2cc623202'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:21.145933 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e4f65cac-983e-486e-af27-76a2cc623202 admin admin] The token cache is being invalidate because project 803d0eb4f2d04a87ae4d5b16bcf2aef9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:59:21.145933 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e4f65cac-983e-486e-af27-76a2cc623202 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:21.146859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e4f65cac-983e-486e-af27-76a2cc623202 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:21.149139 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1204/2392] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:59:20 2022] DELETE /identity/v3/projects/803d0eb4f2d04a87ae4d5b16bcf2aef9 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:23.299375 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53587623-63b0-4c21-8478-ea4a515db8fe None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:23.299968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53587623-63b0-4c21-8478-ea4a515db8fe None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:23.299968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53587623-63b0-4c21-8478-ea4a515db8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:23.319513 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-53587623-63b0-4c21-8478-ea4a515db8fe None None] MFA Rules not processed for user `97011bf8f6d444468fecdde7acfa1020`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:23.320865 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-53587623-63b0-4c21-8478-ea4a515db8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:23.379727 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1189/2393] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:59:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:27.296383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f65cac-983e-486e-af27-76a2cc623202 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:27.297959 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e4f65cac-983e-486e-af27-76a2cc623202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:27.413737 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f65cac-983e-486e-af27-76a2cc623202 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:27.419592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7eafc-63bb-47af-83f2-653a4c80e6c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63789c0a113447ff90933d447cf5bac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:27.420297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7eafc-63bb-47af-83f2-653a4c80e6c4 service placement] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:27.420595 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7eafc-63bb-47af-83f2-653a4c80e6c4 service placement] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:27.420875 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7eafc-63bb-47af-83f2-653a4c80e6c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:27.422931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-fbc7eafc-63bb-47af-83f2-653a4c80e6c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:27.443547 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc7eafc-63bb-47af-83f2-653a4c80e6c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:27.447299 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc7eafc-63bb-47af-83f2-653a4c80e6c4 service placement] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:27.461587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1205/2394] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:59:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:33.371595 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:DELETE=8,keystone:SELECT=45,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:37.456830 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:DELETE=7,keystone:SELECT=56,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:51.008692 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3f6a7-efdc-4f11-ab21-df2bc23d279c None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:51.009271 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3f6a7-efdc-4f11-ab21-df2bc23d279c None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:51.009271 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3f6a7-efdc-4f11-ab21-df2bc23d279c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:51.028539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-2ce3f6a7-efdc-4f11-ab21-df2bc23d279c None None] MFA Rules not processed for user `daec78050ee547699910603eff8cb7ca`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:51.029766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-2ce3f6a7-efdc-4f11-ab21-df2bc23d279c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:51.094602 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1190/2395] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:59:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:59:53.434393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7eafc-63bb-47af-83f2-653a4c80e6c4 service placement] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:53.438601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-fbc7eafc-63bb-47af-83f2-653a4c80e6c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:53.536899 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7eafc-63bb-47af-83f2-653a4c80e6c4 service placement] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:53.541624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf62a53-0b3c-4f21-a6a0-a4838b20510f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:53.542616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf62a53-0b3c-4f21-a6a0-a4838b20510f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:53.542899 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf62a53-0b3c-4f21-a6a0-a4838b20510f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:53.543179 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf62a53-0b3c-4f21-a6a0-a4838b20510f admin admin] PATH_INFO: `/v3/users/8a7c58d2cd0347fcb07725493c9b3316` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:53.544425 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf62a53-0b3c-4f21-a6a0-a4838b20510f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a7c58d2cd0347fcb07725493c9b3316)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:53.546436 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf62a53-0b3c-4f21-a6a0-a4838b20510f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:53.605959 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-cdf62a53-0b3c-4f21-a6a0-a4838b20510f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a7c58d2cd0347fcb07725493c9b3316'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:53.622643 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-cdf62a53-0b3c-4f21-a6a0-a4838b20510f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a7c58d2cd0347fcb07725493c9b3316'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:53.633889 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-cdf62a53-0b3c-4f21-a6a0-a4838b20510f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a7c58d2cd0347fcb07725493c9b3316'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:53.635138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-cdf62a53-0b3c-4f21-a6a0-a4838b20510f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a7c58d2cd0347fcb07725493c9b3316'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:53.635949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-cdf62a53-0b3c-4f21-a6a0-a4838b20510f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a7c58d2cd0347fcb07725493c9b3316'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:53.652888 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1206/2396] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 14:59:53 2022] DELETE /identity/v3/users/8a7c58d2cd0347fcb07725493c9b3316 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:59:53.657682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dcce3-8fe6-4c1d-888c-6de682fc5d0e None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:53.662152 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dcce3-8fe6-4c1d-888c-6de682fc5d0e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:53.663665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dcce3-8fe6-4c1d-888c-6de682fc5d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:53.684368 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-b61dcce3-8fe6-4c1d-888c-6de682fc5d0e None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:59:53.686141 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-b61dcce3-8fe6-4c1d-888c-6de682fc5d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:53.820732 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1191/2397] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 14:59:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:59:53.833405 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf62a53-0b3c-4f21-a6a0-a4838b20510f admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:53.836935 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-cdf62a53-0b3c-4f21-a6a0-a4838b20510f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:53.972250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf62a53-0b3c-4f21-a6a0-a4838b20510f admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:53.976991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf510f7f-d4d5-414b-8b22-36d6a9f039d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:53.978165 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf510f7f-d4d5-414b-8b22-36d6a9f039d1 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:53.978583 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf510f7f-d4d5-414b-8b22-36d6a9f039d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:53.979131 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf510f7f-d4d5-414b-8b22-36d6a9f039d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:53.990300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf510f7f-d4d5-414b-8b22-36d6a9f039d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:53.991824 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf510f7f-d4d5-414b-8b22-36d6a9f039d1 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:54.003074 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1207/2398] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 14:59:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:54.344425 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61dcce3-8fe6-4c1d-888c-6de682fc5d0e None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:54.346186 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-b61dcce3-8fe6-4c1d-888c-6de682fc5d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:54.370474 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61dcce3-8fe6-4c1d-888c-6de682fc5d0e None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:54.376514 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fed455-a81c-4dae-9c3f-56ec13845bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:54.378174 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fed455-a81c-4dae-9c3f-56ec13845bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:54.378924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fed455-a81c-4dae-9c3f-56ec13845bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:54.380203 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fed455-a81c-4dae-9c3f-56ec13845bf6 admin admin] PATH_INFO: `/v3/projects/7e6a9b5901d94962a7df33f91ed76715` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:54.384088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fed455-a81c-4dae-9c3f-56ec13845bf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e6a9b5901d94962a7df33f91ed76715)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:54.388336 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fed455-a81c-4dae-9c3f-56ec13845bf6 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:54.457312 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-92fed455-a81c-4dae-9c3f-56ec13845bf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e6a9b5901d94962a7df33f91ed76715', 'request_id': 'req-92fed455-a81c-4dae-9c3f-56ec13845bf6'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:54.469604 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-92fed455-a81c-4dae-9c3f-56ec13845bf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e6a9b5901d94962a7df33f91ed76715', 'request_id': 'req-92fed455-a81c-4dae-9c3f-56ec13845bf6'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:54.481111 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-92fed455-a81c-4dae-9c3f-56ec13845bf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e6a9b5901d94962a7df33f91ed76715', 'request_id': 'req-92fed455-a81c-4dae-9c3f-56ec13845bf6'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:54.517804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-92fed455-a81c-4dae-9c3f-56ec13845bf6 admin admin] The token cache is being invalidate because project 7e6a9b5901d94962a7df33f91ed76715 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:59:54.518266 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-92fed455-a81c-4dae-9c3f-56ec13845bf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:54.519339 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-92fed455-a81c-4dae-9c3f-56ec13845bf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:59:54.521459 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1192/2399] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 14:59:54 2022] DELETE /identity/v3/projects/7e6a9b5901d94962a7df33f91ed76715 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:00:03.997882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:00:04.510144 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:00:10.981179 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf510f7f-d4d5-414b-8b22-36d6a9f039d1 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:00:10.990653 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf510f7f-d4d5-414b-8b22-36d6a9f039d1 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:00:11.044379 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216eab7-0766-40a3-8f94-98393357852e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:00:11.045047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216eab7-0766-40a3-8f94-98393357852e service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:00:11.045333 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216eab7-0766-40a3-8f94-98393357852e service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:00:11.045641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216eab7-0766-40a3-8f94-98393357852e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:00:11.047712 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a216eab7-0766-40a3-8f94-98393357852e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:00:11.113095 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216eab7-0766-40a3-8f94-98393357852e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:00:11.114592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216eab7-0766-40a3-8f94-98393357852e service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:00:11.125439 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1208/2400] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:00:10 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:00:21.120340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=50 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:01:02.899643 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fed455-a81c-4dae-9c3f-56ec13845bf6 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:01:02.901147 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-92fed455-a81c-4dae-9c3f-56ec13845bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:01:02.969949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fed455-a81c-4dae-9c3f-56ec13845bf6 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:01:02.973961 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf7c00-3b1d-4c5f-9a24-04dc0ecfc639 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:01:02.975404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf7c00-3b1d-4c5f-9a24-04dc0ecfc639 service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:01:02.975725 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf7c00-3b1d-4c5f-9a24-04dc0ecfc639 service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:01:02.976064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf7c00-3b1d-4c5f-9a24-04dc0ecfc639 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:01:02.978148 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-beaf7c00-3b1d-4c5f-9a24-04dc0ecfc639 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:01:03.059612 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf7c00-3b1d-4c5f-9a24-04dc0ecfc639 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:01:03.062485 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf7c00-3b1d-4c5f-9a24-04dc0ecfc639 service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:01:03.084304 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1193/2401] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:01:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:01:03.277871 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216eab7-0766-40a3-8f94-98393357852e service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:01:03.279421 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a216eab7-0766-40a3-8f94-98393357852e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:01:03.290544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216eab7-0766-40a3-8f94-98393357852e service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:01:03.294635 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ee7f6-e228-40b2-975e-4d82108d5569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:01:03.295291 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ee7f6-e228-40b2-975e-4d82108d5569 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:01:03.295579 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ee7f6-e228-40b2-975e-4d82108d5569 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:01:03.295860 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ee7f6-e228-40b2-975e-4d82108d5569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:01:03.297660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-538ee7f6-e228-40b2-975e-4d82108d5569 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:01:03.315055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ee7f6-e228-40b2-975e-4d82108d5569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:01:03.316523 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ee7f6-e228-40b2-975e-4d82108d5569 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:01:03.328903 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1209/2402] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:01:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:01:12.378530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac5d7a-6644-4e20-83a1-2ac408f01def None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:01:12.379090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac5d7a-6644-4e20-83a1-2ac408f01def None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:01:12.379090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac5d7a-6644-4e20-83a1-2ac408f01def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:01:12.397445 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-d7ac5d7a-6644-4e20-83a1-2ac408f01def None None] MFA Rules not processed for user `e60a5e751f22450c8e366c85b37af1f6`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:01:12.398693 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d7ac5d7a-6644-4e20-83a1-2ac408f01def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:01:12.457709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1194/2403] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:01:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:01:13.160674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ee7f6-e228-40b2-975e-4d82108d5569 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:01:13.162901 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-538ee7f6-e228-40b2-975e-4d82108d5569 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:01:13.236023 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ee7f6-e228-40b2-975e-4d82108d5569 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:01:13.241286 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c5c8d-197e-46e9-867b-c4dcc7de7f91 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63789c0a113447ff90933d447cf5bac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:01:13.241973 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c5c8d-197e-46e9-867b-c4dcc7de7f91 service placement] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:01:13.242290 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c5c8d-197e-46e9-867b-c4dcc7de7f91 service placement] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:01:13.242571 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c5c8d-197e-46e9-867b-c4dcc7de7f91 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:01:13.244408 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-434c5c8d-197e-46e9-867b-c4dcc7de7f91 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:01:13.257034 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c5c8d-197e-46e9-867b-c4dcc7de7f91 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:01:13.258677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c5c8d-197e-46e9-867b-c4dcc7de7f91 service placement] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:01:13.270030 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1210/2404] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:01:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:01:17.758039 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac5d7a-6644-4e20-83a1-2ac408f01def None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:01:17.759783 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d7ac5d7a-6644-4e20-83a1-2ac408f01def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:01:17.776857 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac5d7a-6644-4e20-83a1-2ac408f01def None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:01:17.782522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433c18d-57f8-4130-bc3a-74f3a44cb934 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:01:17.783175 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433c18d-57f8-4130-bc3a-74f3a44cb934 service nova] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:01:17.783454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433c18d-57f8-4130-bc3a-74f3a44cb934 service nova] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:01:17.783730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433c18d-57f8-4130-bc3a-74f3a44cb934 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:01:17.785564 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-6433c18d-57f8-4130-bc3a-74f3a44cb934 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:01:17.830431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:01:17.845773 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6433c18d-57f8-4130-bc3a-74f3a44cb934 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:01:17.847418 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6433c18d-57f8-4130-bc3a-74f3a44cb934 service nova] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:01:17.856977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1195/2405] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:01:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:01:22.988329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63af15c-ec8f-46fc-a190-bce0f970b247 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:01:22.989479 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63af15c-ec8f-46fc-a190-bce0f970b247 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:01:22.989479 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63af15c-ec8f-46fc-a190-bce0f970b247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:01:23.009571 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b63af15c-ec8f-46fc-a190-bce0f970b247 None None] MFA Rules not processed for user `97011bf8f6d444468fecdde7acfa1020`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:01:23.010846 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b63af15c-ec8f-46fc-a190-bce0f970b247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:01:23.021853 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1211/2406] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:01:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:01:27.852638 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=7 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:01:28.542487 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433c18d-57f8-4130-bc3a-74f3a44cb934 service nova] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:01:28.552674 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433c18d-57f8-4130-bc3a-74f3a44cb934 service nova] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:01:28.556419 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d868a-f9d6-42c7-9c7d-a20e55c6d58e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:01:28.557075 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d868a-f9d6-42c7-9c7d-a20e55c6d58e service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:01:28.557359 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d868a-f9d6-42c7-9c7d-a20e55c6d58e service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:01:28.557635 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d868a-f9d6-42c7-9c7d-a20e55c6d58e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:01:28.559537 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-6f1d868a-f9d6-42c7-9c7d-a20e55c6d58e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:01:28.569905 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d868a-f9d6-42c7-9c7d-a20e55c6d58e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:01:28.571395 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d868a-f9d6-42c7-9c7d-a20e55c6d58e service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:01:28.580578 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1196/2407] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:01:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:01:32.916643 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63af15c-ec8f-46fc-a190-bce0f970b247 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:01:32.924456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=42 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:01:32.931501 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63af15c-ec8f-46fc-a190-bce0f970b247 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:01:32.937231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf44452-cffb-4a01-a518-cd0bd08bbc60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:01:32.937878 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf44452-cffb-4a01-a518-cd0bd08bbc60 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:01:32.938200 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf44452-cffb-4a01-a518-cd0bd08bbc60 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:01:32.938482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf44452-cffb-4a01-a518-cd0bd08bbc60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:01:32.940262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3bf44452-cffb-4a01-a518-cd0bd08bbc60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:01:32.953377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf44452-cffb-4a01-a518-cd0bd08bbc60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:01:32.955141 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf44452-cffb-4a01-a518-cd0bd08bbc60 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:01:32.967743 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1212/2408] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:01:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:01:38.576345 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=6 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:01:42.962560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:01:47.655953 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d868a-f9d6-42c7-9c7d-a20e55c6d58e service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:01:47.665482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d868a-f9d6-42c7-9c7d-a20e55c6d58e service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:01:47.669062 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7dfb47-90bc-439a-878a-cdc9d8fd012c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:01:47.669707 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7dfb47-90bc-439a-878a-cdc9d8fd012c service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:01:47.669981 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7dfb47-90bc-439a-878a-cdc9d8fd012c service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:01:47.670289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7dfb47-90bc-439a-878a-cdc9d8fd012c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:01:47.672101 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-0a7dfb47-90bc-439a-878a-cdc9d8fd012c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:01:47.734219 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7dfb47-90bc-439a-878a-cdc9d8fd012c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:01:47.735670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7dfb47-90bc-439a-878a-cdc9d8fd012c service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:01:47.745629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1197/2409] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:01:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:01:54.922193 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d079c-30e7-4645-8c72-607ff007130a None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:01:54.922680 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d079c-30e7-4645-8c72-607ff007130a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:01:54.922798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d079c-30e7-4645-8c72-607ff007130a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:01:54.940087 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-6d7d079c-30e7-4645-8c72-607ff007130a None None] MFA Rules not processed for user `7fedb9b92e9a4e3cb671f69751e426a1`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:01:54.941289 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-6d7d079c-30e7-4645-8c72-607ff007130a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:01:54.948944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1213/2410] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:01:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:01:55.245398 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7dfb47-90bc-439a-878a-cdc9d8fd012c service cinder] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:01:55.246962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-0a7dfb47-90bc-439a-878a-cdc9d8fd012c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:01:55.259388 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7dfb47-90bc-439a-878a-cdc9d8fd012c service cinder] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:01:55.263384 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0672ada-4e1f-47ee-86c6-b984ad1cf5b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:01:55.264045 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0672ada-4e1f-47ee-86c6-b984ad1cf5b2 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:01:55.264336 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0672ada-4e1f-47ee-86c6-b984ad1cf5b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:01:55.264607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0672ada-4e1f-47ee-86c6-b984ad1cf5b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:01:55.266446 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-e0672ada-4e1f-47ee-86c6-b984ad1cf5b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:01:55.276188 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0672ada-4e1f-47ee-86c6-b984ad1cf5b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:01:55.277623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0672ada-4e1f-47ee-86c6-b984ad1cf5b2 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:01:55.285781 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1198/2411] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:01:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:02:04.935288 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:05.282141 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=34 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:05.789166 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d079c-30e7-4645-8c72-607ff007130a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:05.804317 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d079c-30e7-4645-8c72-607ff007130a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:05.810131 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c73843-0e37-4f97-adc9-ac1e43082b32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:05.810805 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c73843-0e37-4f97-adc9-ac1e43082b32 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:05.811087 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c73843-0e37-4f97-adc9-ac1e43082b32 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:05.811364 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c73843-0e37-4f97-adc9-ac1e43082b32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:05.813255 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e6c73843-0e37-4f97-adc9-ac1e43082b32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:05.826965 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c73843-0e37-4f97-adc9-ac1e43082b32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:05.828600 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c73843-0e37-4f97-adc9-ac1e43082b32 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:05.841144 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1214/2412] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:02:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:02:10.113336 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0672ada-4e1f-47ee-86c6-b984ad1cf5b2 service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:10.115606 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-e0672ada-4e1f-47ee-86c6-b984ad1cf5b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:10.134880 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0672ada-4e1f-47ee-86c6-b984ad1cf5b2 service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:10.140503 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a5f5a-6155-4fda-883e-b51c9a38da0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:10.141152 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a5f5a-6155-4fda-883e-b51c9a38da0c service nova] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:10.141434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a5f5a-6155-4fda-883e-b51c9a38da0c service nova] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:10.141707 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a5f5a-6155-4fda-883e-b51c9a38da0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:10.143528 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-b42a5f5a-6155-4fda-883e-b51c9a38da0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:10.157322 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a5f5a-6155-4fda-883e-b51c9a38da0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:10.159179 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a5f5a-6155-4fda-883e-b51c9a38da0c service nova] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:10.171109 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1199/2413] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:02:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:02:15.835481 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=6 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:20.165617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=6 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:22.280440 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c73843-0e37-4f97-adc9-ac1e43082b32 service nova] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:22.292955 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c73843-0e37-4f97-adc9-ac1e43082b32 service nova] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:22.298801 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5ad9e-7dcc-4ba6-a76e-849cd7b74724 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:22.299455 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5ad9e-7dcc-4ba6-a76e-849cd7b74724 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:22.299740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5ad9e-7dcc-4ba6-a76e-849cd7b74724 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:22.300017 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5ad9e-7dcc-4ba6-a76e-849cd7b74724 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:22.301818 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9bc5ad9e-7dcc-4ba6-a76e-849cd7b74724 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:22.313104 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc5ad9e-7dcc-4ba6-a76e-849cd7b74724 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:22.315258 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc5ad9e-7dcc-4ba6-a76e-849cd7b74724 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:22.325362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1215/2414] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:02:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:02:32.320697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=6 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:41.328374 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f9db4-5695-43f2-b774-e65ee9b401ee None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:41.329114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f9db4-5695-43f2-b774-e65ee9b401ee None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:41.329114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f9db4-5695-43f2-b774-e65ee9b401ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:41.348253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-bb9f9db4-5695-43f2-b774-e65ee9b401ee None None] MFA Rules not processed for user `daec78050ee547699910603eff8cb7ca`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:02:41.349469 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-bb9f9db4-5695-43f2-b774-e65ee9b401ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:41.414282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1200/2415] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:02:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:02:43.113254 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5ad9e-7dcc-4ba6-a76e-849cd7b74724 service nova] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:43.123092 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5ad9e-7dcc-4ba6-a76e-849cd7b74724 service nova] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:43.126719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53f414-ae0e-4ce8-9970-58f13778a452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:43.127358 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53f414-ae0e-4ce8-9970-58f13778a452 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:43.127834 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53f414-ae0e-4ce8-9970-58f13778a452 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:43.127960 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53f414-ae0e-4ce8-9970-58f13778a452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:43.129833 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-dd53f414-ae0e-4ce8-9970-58f13778a452 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:43.193427 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53f414-ae0e-4ce8-9970-58f13778a452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:43.194808 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53f414-ae0e-4ce8-9970-58f13778a452 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:43.204624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1216/2416] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:02:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:02:48.424447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f9db4-5695-43f2-b774-e65ee9b401ee None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:48.426090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-bb9f9db4-5695-43f2-b774-e65ee9b401ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:48.451178 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f9db4-5695-43f2-b774-e65ee9b401ee None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:48.459795 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921b5dc-99cb-489a-b761-3db4194c278b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63789c0a113447ff90933d447cf5bac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:48.461251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921b5dc-99cb-489a-b761-3db4194c278b service placement] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:48.461973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921b5dc-99cb-489a-b761-3db4194c278b service placement] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:48.462704 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921b5dc-99cb-489a-b761-3db4194c278b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:48.466188 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d921b5dc-99cb-489a-b761-3db4194c278b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:48.489088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921b5dc-99cb-489a-b761-3db4194c278b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:48.491815 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921b5dc-99cb-489a-b761-3db4194c278b service placement] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:48.503365 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53f414-ae0e-4ce8-9970-58f13778a452 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:48.505283 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-dd53f414-ae0e-4ce8-9970-58f13778a452 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:48.520190 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1201/2417] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:02:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:02:48.535615 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53f414-ae0e-4ce8-9970-58f13778a452 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:48.543367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c81ded-0053-4a8e-aa5e-2018687006ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:48.544358 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c81ded-0053-4a8e-aa5e-2018687006ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:48.544711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c81ded-0053-4a8e-aa5e-2018687006ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:48.545056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c81ded-0053-4a8e-aa5e-2018687006ec admin admin] PATH_INFO: `/v3/users/97011bf8f6d444468fecdde7acfa1020` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:48.546694 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c81ded-0053-4a8e-aa5e-2018687006ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=97011bf8f6d444468fecdde7acfa1020)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:48.549601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c81ded-0053-4a8e-aa5e-2018687006ec admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:48.613166 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-04c81ded-0053-4a8e-aa5e-2018687006ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97011bf8f6d444468fecdde7acfa1020'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:48.628457 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-04c81ded-0053-4a8e-aa5e-2018687006ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97011bf8f6d444468fecdde7acfa1020'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:48.638285 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-04c81ded-0053-4a8e-aa5e-2018687006ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97011bf8f6d444468fecdde7acfa1020'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:48.639865 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-04c81ded-0053-4a8e-aa5e-2018687006ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97011bf8f6d444468fecdde7acfa1020'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:48.640560 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-04c81ded-0053-4a8e-aa5e-2018687006ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97011bf8f6d444468fecdde7acfa1020'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:48.656866 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1217/2418] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:02:48 2022] DELETE /identity/v3/users/97011bf8f6d444468fecdde7acfa1020 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:02:49.084814 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921b5dc-99cb-489a-b761-3db4194c278b service placement] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:49.099936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921b5dc-99cb-489a-b761-3db4194c278b service placement] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:49.200785 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb057e-be47-4490-abba-523fc1bb9b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:49.201949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb057e-be47-4490-abba-523fc1bb9b40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:49.202329 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb057e-be47-4490-abba-523fc1bb9b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:49.202674 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb057e-be47-4490-abba-523fc1bb9b40 admin admin] PATH_INFO: `/v3/projects/acd27057280d4bdda5b9a689a1e399ff` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:49.204243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb057e-be47-4490-abba-523fc1bb9b40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acd27057280d4bdda5b9a689a1e399ff)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:49.206765 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb057e-be47-4490-abba-523fc1bb9b40 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:49.268500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-20cb057e-be47-4490-abba-523fc1bb9b40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acd27057280d4bdda5b9a689a1e399ff', 'request_id': 'req-20cb057e-be47-4490-abba-523fc1bb9b40'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:49.275984 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-20cb057e-be47-4490-abba-523fc1bb9b40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acd27057280d4bdda5b9a689a1e399ff', 'request_id': 'req-20cb057e-be47-4490-abba-523fc1bb9b40'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:49.285712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-20cb057e-be47-4490-abba-523fc1bb9b40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acd27057280d4bdda5b9a689a1e399ff', 'request_id': 'req-20cb057e-be47-4490-abba-523fc1bb9b40'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:49.302258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-20cb057e-be47-4490-abba-523fc1bb9b40 admin admin] The token cache is being invalidate because project acd27057280d4bdda5b9a689a1e399ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:02:49.302695 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-20cb057e-be47-4490-abba-523fc1bb9b40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:49.303456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-20cb057e-be47-4490-abba-523fc1bb9b40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:02:49.305112 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1202/2419] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:02:49 2022] DELETE /identity/v3/projects/acd27057280d4bdda5b9a689a1e399ff => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:02:49.310270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af29d6-a2e8-43a6-b581-884ccf89c696 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:49.310587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af29d6-a2e8-43a6-b581-884ccf89c696 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:49.310865 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af29d6-a2e8-43a6-b581-884ccf89c696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:49.331357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-70af29d6-a2e8-43a6-b581-884ccf89c696 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:02:49.332575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-70af29d6-a2e8-43a6-b581-884ccf89c696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:49.409641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1218/2420] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:02:49.418923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde602fc-ad2c-41fd-994f-2e25f66dd802 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:49.419211 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde602fc-ad2c-41fd-994f-2e25f66dd802 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:49.419484 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde602fc-ad2c-41fd-994f-2e25f66dd802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:49.439722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-cde602fc-ad2c-41fd-994f-2e25f66dd802 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:02:49.440958 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-cde602fc-ad2c-41fd-994f-2e25f66dd802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:49.449906 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1203/2421] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:02:49.454146 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af29d6-a2e8-43a6-b581-884ccf89c696 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:49.466717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af29d6-a2e8-43a6-b581-884ccf89c696 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:49.472054 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fd53a-ef83-4e33-8a17-3f49dda10510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:49.472765 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fd53a-ef83-4e33-8a17-3f49dda10510 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:49.473050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fd53a-ef83-4e33-8a17-3f49dda10510 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:49.473330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fd53a-ef83-4e33-8a17-3f49dda10510 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:49.474103 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124fd53a-ef83-4e33-8a17-3f49dda10510 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:02:49.474418 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124fd53a-ef83-4e33-8a17-3f49dda10510 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:49.476075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124fd53a-ef83-4e33-8a17-3f49dda10510 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:49.478288 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:49.498356 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1219/2422] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 15:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:02:49.501399 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde602fc-ad2c-41fd-994f-2e25f66dd802 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:49.512887 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde602fc-ad2c-41fd-994f-2e25f66dd802 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:49.518509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5dbaa-6bcd-45be-af2b-b5daca7ab5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:49.519230 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5dbaa-6bcd-45be-af2b-b5daca7ab5d9 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:49.519507 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5dbaa-6bcd-45be-af2b-b5daca7ab5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:49.519921 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5dbaa-6bcd-45be-af2b-b5daca7ab5d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:49.520953 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5dbaa-6bcd-45be-af2b-b5daca7ab5d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:49.523010 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5dbaa-6bcd-45be-af2b-b5daca7ab5d9 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:49.538714 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:49.569550 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1204/2423] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:02:49 2022] POST /identity/v3/projects => generated 389 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:02:49.575305 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fd53a-ef83-4e33-8a17-3f49dda10510 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:49.586127 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fd53a-ef83-4e33-8a17-3f49dda10510 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:49.656408 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751730cf-ab48-4299-8cf1-2fbd1c57bce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:49.657550 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751730cf-ab48-4299-8cf1-2fbd1c57bce9 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:49.657912 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751730cf-ab48-4299-8cf1-2fbd1c57bce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:49.658323 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751730cf-ab48-4299-8cf1-2fbd1c57bce9 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:49.659148 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751730cf-ab48-4299-8cf1-2fbd1c57bce9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:49.661243 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751730cf-ab48-4299-8cf1-2fbd1c57bce9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:49.686122 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1220/2424] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:02:49 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:02:49.691123 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5dbaa-6bcd-45be-af2b-b5daca7ab5d9 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:49.712805 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5dbaa-6bcd-45be-af2b-b5daca7ab5d9 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:49.722660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c73da-7bca-45ef-9734-6dfc6d6581f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:49.724566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c73da-7bca-45ef-9734-6dfc6d6581f3 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:49.725173 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c73da-7bca-45ef-9734-6dfc6d6581f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:49.725744 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c73da-7bca-45ef-9734-6dfc6d6581f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:49.727243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c73da-7bca-45ef-9734-6dfc6d6581f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:49.730936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c73da-7bca-45ef-9734-6dfc6d6581f3 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:49.751614 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1205/2425] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:02:49.756058 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751730cf-ab48-4299-8cf1-2fbd1c57bce9 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:49.768776 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751730cf-ab48-4299-8cf1-2fbd1c57bce9 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:49.773501 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11dd4e-2a8f-48cf-84d6-c10cbb801b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:49.774201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11dd4e-2a8f-48cf-84d6-c10cbb801b86 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:49.774487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11dd4e-2a8f-48cf-84d6-c10cbb801b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:49.774764 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11dd4e-2a8f-48cf-84d6-c10cbb801b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:49.775491 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11dd4e-2a8f-48cf-84d6-c10cbb801b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:49.777232 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11dd4e-2a8f-48cf-84d6-c10cbb801b86 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:49.795480 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1221/2426] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:02:49.796742 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c73da-7bca-45ef-9734-6dfc6d6581f3 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:49.807241 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c73da-7bca-45ef-9734-6dfc6d6581f3 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:49.812925 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8257c2-d9db-4de5-be64-e79b1a4b21f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:49.813819 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8257c2-d9db-4de5-be64-e79b1a4b21f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:49.814202 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8257c2-d9db-4de5-be64-e79b1a4b21f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:49.814420 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8257c2-d9db-4de5-be64-e79b1a4b21f5 admin admin] PATH_INFO: `/v3/projects/2b586a13e82f4177a7180f81acf675fd/users/34b1591b0eb944ed98c42d35a03c1ff9/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:49.831214 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8257c2-d9db-4de5-be64-e79b1a4b21f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b586a13e82f4177a7180f81acf675fd, user_id=34b1591b0eb944ed98c42d35a03c1ff9, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:49.833906 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8257c2-d9db-4de5-be64-e79b1a4b21f5 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:49.849309 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1206/2427] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:02:49 2022] PUT /identity/v3/projects/2b586a13e82f4177a7180f81acf675fd/users/34b1591b0eb944ed98c42d35a03c1ff9/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:02:49.854636 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24861071-39fc-44cf-b2bb-992cff798bf3 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:49.854997 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24861071-39fc-44cf-b2bb-992cff798bf3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:49.855287 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24861071-39fc-44cf-b2bb-992cff798bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:49.874166 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-24861071-39fc-44cf-b2bb-992cff798bf3 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:02:49.875427 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-24861071-39fc-44cf-b2bb-992cff798bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:49.965486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1222/2428] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:02:49.975612 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8257c2-d9db-4de5-be64-e79b1a4b21f5 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:49.977345 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-6d8257c2-d9db-4de5-be64-e79b1a4b21f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:50.039923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8257c2-d9db-4de5-be64-e79b1a4b21f5 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:50.044306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3ad17-6b74-422f-8ad7-03f5f96e562a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:50.045036 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3ad17-6b74-422f-8ad7-03f5f96e562a service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:50.045254 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3ad17-6b74-422f-8ad7-03f5f96e562a service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:50.045535 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3ad17-6b74-422f-8ad7-03f5f96e562a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:50.055482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb3ad17-6b74-422f-8ad7-03f5f96e562a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:50.056914 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb3ad17-6b74-422f-8ad7-03f5f96e562a service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:50.067391 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1207/2429] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:02:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:02:50.841642 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5d1f6-668a-490e-a2c1-8756956d8f00 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:50.842582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5d1f6-668a-490e-a2c1-8756956d8f00 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:50.842936 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5d1f6-668a-490e-a2c1-8756956d8f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:50.871844 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-21b5d1f6-668a-490e-a2c1-8756956d8f00 None None] MFA Rules not processed for user `e60a5e751f22450c8e366c85b37af1f6`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:02:50.875700 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-21b5d1f6-668a-490e-a2c1-8756956d8f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:50.976838 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1223/2430] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:02:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:02:56.132664 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77093f-e9a4-41e1-a5d2-18f1e99419d6 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:56.133460 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77093f-e9a4-41e1-a5d2-18f1e99419d6 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:56.133570 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77093f-e9a4-41e1-a5d2-18f1e99419d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:56.183685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-fb77093f-e9a4-41e1-a5d2-18f1e99419d6 None None] MFA Rules not processed for user `34b1591b0eb944ed98c42d35a03c1ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:02:56.185463 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-fb77093f-e9a4-41e1-a5d2-18f1e99419d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:56.290564 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1208/2431] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:02:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:02:56.304853 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5d1f6-668a-490e-a2c1-8756956d8f00 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:56.306301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-21b5d1f6-668a-490e-a2c1-8756956d8f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:56.331142 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:56.407152 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5d1f6-668a-490e-a2c1-8756956d8f00 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:56.414127 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e52eff-83ec-4345-826e-a82d4e564c0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:56.415040 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e52eff-83ec-4345-826e-a82d4e564c0b service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:56.415474 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e52eff-83ec-4345-826e-a82d4e564c0b service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:56.415904 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e52eff-83ec-4345-826e-a82d4e564c0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:56.425534 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e52eff-83ec-4345-826e-a82d4e564c0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:56.427265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e52eff-83ec-4345-826e-a82d4e564c0b service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:56.438466 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1224/2432] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:02:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:02:58.637227 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582f8f3-484a-4c9a-bcb5-a0c5003ffc38 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:58.637871 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582f8f3-484a-4c9a-bcb5-a0c5003ffc38 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:58.638633 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582f8f3-484a-4c9a-bcb5-a0c5003ffc38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:58.695542 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-6582f8f3-484a-4c9a-bcb5-a0c5003ffc38 None None] MFA Rules not processed for user `34b1591b0eb944ed98c42d35a03c1ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:02:58.697165 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-6582f8f3-484a-4c9a-bcb5-a0c5003ffc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:58.714691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1209/2433] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:02:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:03:06.436591 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=26 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:08.666434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=82,keystone:INSERT=2 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:34.624677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e52eff-83ec-4345-826e-a82d4e564c0b service nova] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:34.626155 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b1e52eff-83ec-4345-826e-a82d4e564c0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:34.638363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e52eff-83ec-4345-826e-a82d4e564c0b service nova] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:34.642841 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d46b3e-704a-4018-990d-08b8fa73220a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:34.643492 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d46b3e-704a-4018-990d-08b8fa73220a service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:34.643780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d46b3e-704a-4018-990d-08b8fa73220a service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:34.644058 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d46b3e-704a-4018-990d-08b8fa73220a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:34.645807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-13d46b3e-704a-4018-990d-08b8fa73220a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:34.657731 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d46b3e-704a-4018-990d-08b8fa73220a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:34.659287 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d46b3e-704a-4018-990d-08b8fa73220a service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:34.669335 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1225/2434] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:03:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:03:37.822185 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582f8f3-484a-4c9a-bcb5-a0c5003ffc38 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:37.823581 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-6582f8f3-484a-4c9a-bcb5-a0c5003ffc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:37.875358 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d46b3e-704a-4018-990d-08b8fa73220a service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:37.876781 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-13d46b3e-704a-4018-990d-08b8fa73220a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:37.891659 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d46b3e-704a-4018-990d-08b8fa73220a service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:37.898982 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c719d95-d41d-46a7-b404-331e6a3e3975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:37.899880 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c719d95-d41d-46a7-b404-331e6a3e3975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:37.900145 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c719d95-d41d-46a7-b404-331e6a3e3975 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:37.900592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c719d95-d41d-46a7-b404-331e6a3e3975 admin admin] PATH_INFO: `/v3/users/7fedb9b92e9a4e3cb671f69751e426a1` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:37.908915 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c719d95-d41d-46a7-b404-331e6a3e3975 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fedb9b92e9a4e3cb671f69751e426a1)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:37.910651 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c719d95-d41d-46a7-b404-331e6a3e3975 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:37.913127 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582f8f3-484a-4c9a-bcb5-a0c5003ffc38 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:37.919497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80f799-40ff-44f2-89cf-a495c3edf8df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63789c0a113447ff90933d447cf5bac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:37.920178 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80f799-40ff-44f2-89cf-a495c3edf8df service placement] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:37.920472 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80f799-40ff-44f2-89cf-a495c3edf8df service placement] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:37.920745 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80f799-40ff-44f2-89cf-a495c3edf8df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:37.922569 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-1d80f799-40ff-44f2-89cf-a495c3edf8df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:37.934618 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80f799-40ff-44f2-89cf-a495c3edf8df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:37.936253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80f799-40ff-44f2-89cf-a495c3edf8df service placement] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:37.946585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1210/2435] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:03:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:03:37.961985 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3c719d95-d41d-46a7-b404-331e6a3e3975 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fedb9b92e9a4e3cb671f69751e426a1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:37.977136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3c719d95-d41d-46a7-b404-331e6a3e3975 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fedb9b92e9a4e3cb671f69751e426a1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:37.985643 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3c719d95-d41d-46a7-b404-331e6a3e3975 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fedb9b92e9a4e3cb671f69751e426a1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:37.986638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3c719d95-d41d-46a7-b404-331e6a3e3975 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fedb9b92e9a4e3cb671f69751e426a1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:37.987423 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3c719d95-d41d-46a7-b404-331e6a3e3975 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fedb9b92e9a4e3cb671f69751e426a1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:38.014546 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1226/2436] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:03:37 2022] DELETE /identity/v3/users/7fedb9b92e9a4e3cb671f69751e426a1 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:38.195805 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80f799-40ff-44f2-89cf-a495c3edf8df service placement] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:38.207740 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80f799-40ff-44f2-89cf-a495c3edf8df service placement] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:38.280164 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015741b8-e88b-47f4-aa70-c2205e01d257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:38.280939 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015741b8-e88b-47f4-aa70-c2205e01d257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:38.281217 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015741b8-e88b-47f4-aa70-c2205e01d257 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:38.281485 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015741b8-e88b-47f4-aa70-c2205e01d257 admin admin] PATH_INFO: `/v3/projects/2395f31fad3645c19096d648c521bf31` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:38.282686 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015741b8-e88b-47f4-aa70-c2205e01d257 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2395f31fad3645c19096d648c521bf31)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:38.284650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015741b8-e88b-47f4-aa70-c2205e01d257 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:38.333002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-015741b8-e88b-47f4-aa70-c2205e01d257 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2395f31fad3645c19096d648c521bf31', 'request_id': 'req-015741b8-e88b-47f4-aa70-c2205e01d257'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:38.338445 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-015741b8-e88b-47f4-aa70-c2205e01d257 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2395f31fad3645c19096d648c521bf31', 'request_id': 'req-015741b8-e88b-47f4-aa70-c2205e01d257'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:38.344571 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-015741b8-e88b-47f4-aa70-c2205e01d257 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2395f31fad3645c19096d648c521bf31', 'request_id': 'req-015741b8-e88b-47f4-aa70-c2205e01d257'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:38.357360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-015741b8-e88b-47f4-aa70-c2205e01d257 admin admin] The token cache is being invalidate because project 2395f31fad3645c19096d648c521bf31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:03:38.357864 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-015741b8-e88b-47f4-aa70-c2205e01d257 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:38.358524 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-015741b8-e88b-47f4-aa70-c2205e01d257 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:38.360145 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1211/2437] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:03:38 2022] DELETE /identity/v3/projects/2395f31fad3645c19096d648c521bf31 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:38.365063 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e219e26-36e7-494b-9109-c09cb15a3bb5 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:38.365385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e219e26-36e7-494b-9109-c09cb15a3bb5 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:38.365669 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e219e26-36e7-494b-9109-c09cb15a3bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:38.385107 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-6e219e26-36e7-494b-9109-c09cb15a3bb5 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:38.386354 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-6e219e26-36e7-494b-9109-c09cb15a3bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:38.448158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1227/2438] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:03:38.455768 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2134b90-878b-4fad-93ef-bf94466a7e76 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:38.456053 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2134b90-878b-4fad-93ef-bf94466a7e76 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:38.456334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2134b90-878b-4fad-93ef-bf94466a7e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:38.474688 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-a2134b90-878b-4fad-93ef-bf94466a7e76 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:38.475889 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-a2134b90-878b-4fad-93ef-bf94466a7e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:38.484442 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1212/2439] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:03:38.488634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e219e26-36e7-494b-9109-c09cb15a3bb5 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:38.499206 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e219e26-36e7-494b-9109-c09cb15a3bb5 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:38.503965 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24495f4-8748-47bf-b34c-a5a6ff5492ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:38.504669 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24495f4-8748-47bf-b34c-a5a6ff5492ac admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:38.504952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24495f4-8748-47bf-b34c-a5a6ff5492ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:38.505230 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24495f4-8748-47bf-b34c-a5a6ff5492ac admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:38.505949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24495f4-8748-47bf-b34c-a5a6ff5492ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:03:38.506289 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24495f4-8748-47bf-b34c-a5a6ff5492ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:38.507960 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24495f4-8748-47bf-b34c-a5a6ff5492ac admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:38.520023 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1228/2440] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 15:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:03:38.524374 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2134b90-878b-4fad-93ef-bf94466a7e76 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:38.533794 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2134b90-878b-4fad-93ef-bf94466a7e76 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:38.538439 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b60c86-b36f-4d3f-a0d9-dc283eb9ee0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:38.539088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b60c86-b36f-4d3f-a0d9-dc283eb9ee0f admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:38.539371 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b60c86-b36f-4d3f-a0d9-dc283eb9ee0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:38.539644 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b60c86-b36f-4d3f-a0d9-dc283eb9ee0f admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:38.540342 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b60c86-b36f-4d3f-a0d9-dc283eb9ee0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:38.542271 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b60c86-b36f-4d3f-a0d9-dc283eb9ee0f admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:38.553125 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:38.570998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1213/2441] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:03:38 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:03:38.575144 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24495f4-8748-47bf-b34c-a5a6ff5492ac admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:38.585389 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24495f4-8748-47bf-b34c-a5a6ff5492ac admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:38.634266 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:38.648354 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d042e-234b-4627-98c0-43efb950644e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:38.649027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d042e-234b-4627-98c0-43efb950644e admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:38.649308 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d042e-234b-4627-98c0-43efb950644e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:38.649585 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d042e-234b-4627-98c0-43efb950644e admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:38.650317 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8d042e-234b-4627-98c0-43efb950644e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:38.652262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8d042e-234b-4627-98c0-43efb950644e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:38.672815 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1229/2442] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:03:38 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:03:38.677101 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b60c86-b36f-4d3f-a0d9-dc283eb9ee0f admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:38.688587 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b60c86-b36f-4d3f-a0d9-dc283eb9ee0f admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:38.693496 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00787fdd-02df-4b9b-9acd-e6ae35d82571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:38.694236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00787fdd-02df-4b9b-9acd-e6ae35d82571 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:38.694656 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00787fdd-02df-4b9b-9acd-e6ae35d82571 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:38.695010 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00787fdd-02df-4b9b-9acd-e6ae35d82571 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:38.695753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00787fdd-02df-4b9b-9acd-e6ae35d82571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:38.697362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00787fdd-02df-4b9b-9acd-e6ae35d82571 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:38.710819 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1214/2443] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:03:38.714572 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d042e-234b-4627-98c0-43efb950644e admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:38.725106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d042e-234b-4627-98c0-43efb950644e admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:38.732330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cab876-959b-4d08-8e5f-a61ca9cbd278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:38.733587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cab876-959b-4d08-8e5f-a61ca9cbd278 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:38.734002 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cab876-959b-4d08-8e5f-a61ca9cbd278 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:38.734335 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cab876-959b-4d08-8e5f-a61ca9cbd278 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:38.735077 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cab876-959b-4d08-8e5f-a61ca9cbd278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:38.736712 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cab876-959b-4d08-8e5f-a61ca9cbd278 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:38.748220 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1230/2444] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:03:38.753169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00787fdd-02df-4b9b-9acd-e6ae35d82571 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:38.763949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00787fdd-02df-4b9b-9acd-e6ae35d82571 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:38.768921 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591d54c-3812-4692-a931-4bcb97f1a681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:38.769629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591d54c-3812-4692-a931-4bcb97f1a681 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:38.769903 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591d54c-3812-4692-a931-4bcb97f1a681 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:38.770217 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591d54c-3812-4692-a931-4bcb97f1a681 admin admin] PATH_INFO: `/v3/projects/fa0aaf9cdfcd49a9bd0578411989ac20/users/2b9a3c0ee7404ec0ac8bb4626afcde64/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:38.786609 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5591d54c-3812-4692-a931-4bcb97f1a681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa0aaf9cdfcd49a9bd0578411989ac20, user_id=2b9a3c0ee7404ec0ac8bb4626afcde64, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:38.789181 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5591d54c-3812-4692-a931-4bcb97f1a681 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:38.803157 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1215/2445] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:03:38 2022] PUT /identity/v3/projects/fa0aaf9cdfcd49a9bd0578411989ac20/users/2b9a3c0ee7404ec0ac8bb4626afcde64/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:38.815710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216aae6b-b677-4e36-a0a9-0a8f59198d09 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:38.815710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216aae6b-b677-4e36-a0a9-0a8f59198d09 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:38.815710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216aae6b-b677-4e36-a0a9-0a8f59198d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:38.840961 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-216aae6b-b677-4e36-a0a9-0a8f59198d09 None None] MFA Rules not processed for user `2b9a3c0ee7404ec0ac8bb4626afcde64`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:38.842791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-216aae6b-b677-4e36-a0a9-0a8f59198d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:38.906464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1231/2446] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:03:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:03:38.910937 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591d54c-3812-4692-a931-4bcb97f1a681 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:38.973421 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591d54c-3812-4692-a931-4bcb97f1a681 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:38.978137 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c7e8d-96db-4a84-b377-bfef78cc49da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:38.978795 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c7e8d-96db-4a84-b377-bfef78cc49da admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:38.979075 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c7e8d-96db-4a84-b377-bfef78cc49da admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:38.979348 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c7e8d-96db-4a84-b377-bfef78cc49da admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:38.980032 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c7e8d-96db-4a84-b377-bfef78cc49da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:38.981936 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c7e8d-96db-4a84-b377-bfef78cc49da admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:39.007555 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1216/2447] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:03:38 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:03:39.011960 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216aae6b-b677-4e36-a0a9-0a8f59198d09 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:39.022883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216aae6b-b677-4e36-a0a9-0a8f59198d09 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:39.079638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba83573-6f5f-4dc6-9c10-fbd16d624ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:39.080230 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba83573-6f5f-4dc6-9c10-fbd16d624ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:39.080516 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba83573-6f5f-4dc6-9c10-fbd16d624ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:39.080793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba83573-6f5f-4dc6-9c10-fbd16d624ee3 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:39.081482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba83573-6f5f-4dc6-9c10-fbd16d624ee3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:39.083460 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba83573-6f5f-4dc6-9c10-fbd16d624ee3 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:39.104028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1232/2448] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:03:39 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:03:39.108550 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c7e8d-96db-4a84-b377-bfef78cc49da admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:39.119530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c7e8d-96db-4a84-b377-bfef78cc49da admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:39.124456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ef523-ec0b-4883-8e91-366ccbf8d51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:39.125120 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ef523-ec0b-4883-8e91-366ccbf8d51d admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:39.125403 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ef523-ec0b-4883-8e91-366ccbf8d51d admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:39.125677 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ef523-ec0b-4883-8e91-366ccbf8d51d admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:39.126435 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ef523-ec0b-4883-8e91-366ccbf8d51d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:39.128056 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ef523-ec0b-4883-8e91-366ccbf8d51d admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:39.139463 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1217/2449] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:03:39.143668 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba83573-6f5f-4dc6-9c10-fbd16d624ee3 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:39.153858 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba83573-6f5f-4dc6-9c10-fbd16d624ee3 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:39.158890 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e3e07-d255-4661-a64c-e19b067cb446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:39.159763 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e3e07-d255-4661-a64c-e19b067cb446 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:39.160050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e3e07-d255-4661-a64c-e19b067cb446 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:39.160332 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e3e07-d255-4661-a64c-e19b067cb446 admin admin] PATH_INFO: `/v3/projects/30ae3fba388b4e13ad079a244e424587/users/8413ba6618ad4db09b8c401cb19ebe9b/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:39.176002 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e3e07-d255-4661-a64c-e19b067cb446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30ae3fba388b4e13ad079a244e424587, user_id=8413ba6618ad4db09b8c401cb19ebe9b, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:39.178868 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e3e07-d255-4661-a64c-e19b067cb446 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:39.192774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1233/2450] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:03:39 2022] PUT /identity/v3/projects/30ae3fba388b4e13ad079a244e424587/users/8413ba6618ad4db09b8c401cb19ebe9b/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:39.196804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ef523-ec0b-4883-8e91-366ccbf8d51d admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:39.207325 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ef523-ec0b-4883-8e91-366ccbf8d51d admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:39.262365 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85f006-1897-4fc5-ad60-99acfff4bde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:39.263029 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85f006-1897-4fc5-ad60-99acfff4bde6 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:39.263312 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85f006-1897-4fc5-ad60-99acfff4bde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:39.263588 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85f006-1897-4fc5-ad60-99acfff4bde6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:39.264824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c85f006-1897-4fc5-ad60-99acfff4bde6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:39.266532 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c85f006-1897-4fc5-ad60-99acfff4bde6 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:39.276355 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1218/2451] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:03:39.281182 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e3e07-d255-4661-a64c-e19b067cb446 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:39.291861 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e3e07-d255-4661-a64c-e19b067cb446 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:39.296716 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b29d30-6ec7-48a8-b23b-f090b40f9a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:39.297599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b29d30-6ec7-48a8-b23b-f090b40f9a43 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:39.297883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b29d30-6ec7-48a8-b23b-f090b40f9a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:39.298191 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b29d30-6ec7-48a8-b23b-f090b40f9a43 admin admin] PATH_INFO: `/v3/projects/30ae3fba388b4e13ad079a244e424587/users/8413ba6618ad4db09b8c401cb19ebe9b/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:39.300202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b29d30-6ec7-48a8-b23b-f090b40f9a43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30ae3fba388b4e13ad079a244e424587, user_id=8413ba6618ad4db09b8c401cb19ebe9b, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:39.302912 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b29d30-6ec7-48a8-b23b-f090b40f9a43 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:39.313084 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1234/2452] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:03:39 2022] PUT /identity/v3/projects/30ae3fba388b4e13ad079a244e424587/users/8413ba6618ad4db09b8c401cb19ebe9b/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:39.323382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17472924-93bb-425c-8d14-5c90528d54d1 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:39.323674 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17472924-93bb-425c-8d14-5c90528d54d1 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:39.323954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17472924-93bb-425c-8d14-5c90528d54d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:39.351076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-17472924-93bb-425c-8d14-5c90528d54d1 None None] MFA Rules not processed for user `8413ba6618ad4db09b8c401cb19ebe9b`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:39.352211 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-17472924-93bb-425c-8d14-5c90528d54d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:39.364859 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:39.427324 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1219/2453] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:03:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:03:39.437972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b29d30-6ec7-48a8-b23b-f090b40f9a43 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:39.439706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f5b29d30-6ec7-48a8-b23b-f090b40f9a43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:39.449624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:39.501254 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b29d30-6ec7-48a8-b23b-f090b40f9a43 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:39.504958 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdbf4a2-5d59-4598-9ec7-8ef02aa62512 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:39.505613 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbf4a2-5d59-4598-9ec7-8ef02aa62512 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:39.505898 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbf4a2-5d59-4598-9ec7-8ef02aa62512 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:39.506203 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbf4a2-5d59-4598-9ec7-8ef02aa62512 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:39.516125 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdbf4a2-5d59-4598-9ec7-8ef02aa62512 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:39.517566 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdbf4a2-5d59-4598-9ec7-8ef02aa62512 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:39.527139 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1235/2454] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:03:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:03:39.598117 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17472924-93bb-425c-8d14-5c90528d54d1 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:39.609172 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17472924-93bb-425c-8d14-5c90528d54d1 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:39.612811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c051a4-3c53-4b91-9425-c0e6e79b9a4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:39.613466 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c051a4-3c53-4b91-9425-c0e6e79b9a4b service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:39.613747 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c051a4-3c53-4b91-9425-c0e6e79b9a4b service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:39.614045 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c051a4-3c53-4b91-9425-c0e6e79b9a4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:39.671298 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c051a4-3c53-4b91-9425-c0e6e79b9a4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:39.672766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c051a4-3c53-4b91-9425-c0e6e79b9a4b service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:39.681741 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1220/2455] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:03:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:03:39.954039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdbf4a2-5d59-4598-9ec7-8ef02aa62512 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:40.016844 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdbf4a2-5d59-4598-9ec7-8ef02aa62512 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:40.021473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd050296-96ae-49fd-bbdb-34d1429c105b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:40.022367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd050296-96ae-49fd-bbdb-34d1429c105b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:40.022649 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd050296-96ae-49fd-bbdb-34d1429c105b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:40.022926 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd050296-96ae-49fd-bbdb-34d1429c105b admin admin] PATH_INFO: `/v3/users/2b9a3c0ee7404ec0ac8bb4626afcde64` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:40.024113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd050296-96ae-49fd-bbdb-34d1429c105b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b9a3c0ee7404ec0ac8bb4626afcde64)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:40.026071 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd050296-96ae-49fd-bbdb-34d1429c105b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:40.072836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fd050296-96ae-49fd-bbdb-34d1429c105b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b9a3c0ee7404ec0ac8bb4626afcde64'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:40.084956 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fd050296-96ae-49fd-bbdb-34d1429c105b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b9a3c0ee7404ec0ac8bb4626afcde64'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:40.092713 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fd050296-96ae-49fd-bbdb-34d1429c105b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b9a3c0ee7404ec0ac8bb4626afcde64'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:40.093485 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fd050296-96ae-49fd-bbdb-34d1429c105b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b9a3c0ee7404ec0ac8bb4626afcde64'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:40.094136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fd050296-96ae-49fd-bbdb-34d1429c105b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b9a3c0ee7404ec0ac8bb4626afcde64'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:40.106599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1236/2456] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:03:39 2022] DELETE /identity/v3/users/2b9a3c0ee7404ec0ac8bb4626afcde64 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:40.117195 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005e3f2-ea58-4d18-8874-fbacd694ae41 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:40.117722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005e3f2-ea58-4d18-8874-fbacd694ae41 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:40.117722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005e3f2-ea58-4d18-8874-fbacd694ae41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:40.137363 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-e005e3f2-ea58-4d18-8874-fbacd694ae41 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:40.138608 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-e005e3f2-ea58-4d18-8874-fbacd694ae41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:40.197910 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1221/2457] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:03:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:03:40.208052 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd050296-96ae-49fd-bbdb-34d1429c105b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:40.209537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-fd050296-96ae-49fd-bbdb-34d1429c105b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:40.241756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:40.270165 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd050296-96ae-49fd-bbdb-34d1429c105b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:40.273773 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52956502-acee-4a8a-9060-1cd1d5b3ea77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:40.274455 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52956502-acee-4a8a-9060-1cd1d5b3ea77 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:40.274738 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52956502-acee-4a8a-9060-1cd1d5b3ea77 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:40.275015 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52956502-acee-4a8a-9060-1cd1d5b3ea77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:40.284692 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52956502-acee-4a8a-9060-1cd1d5b3ea77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:40.286170 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52956502-acee-4a8a-9060-1cd1d5b3ea77 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:40.296591 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1237/2458] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:03:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:03:40.577038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005e3f2-ea58-4d18-8874-fbacd694ae41 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:40.579146 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-e005e3f2-ea58-4d18-8874-fbacd694ae41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:40.596278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005e3f2-ea58-4d18-8874-fbacd694ae41 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:40.602726 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3308a4-8ff4-48ad-a465-803afd955615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:40.603897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3308a4-8ff4-48ad-a465-803afd955615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:40.604275 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3308a4-8ff4-48ad-a465-803afd955615 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:40.604585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3308a4-8ff4-48ad-a465-803afd955615 admin admin] PATH_INFO: `/v3/projects/fa0aaf9cdfcd49a9bd0578411989ac20` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:40.606298 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3308a4-8ff4-48ad-a465-803afd955615 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa0aaf9cdfcd49a9bd0578411989ac20)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:40.609064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3308a4-8ff4-48ad-a465-803afd955615 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:40.653432 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:40.657528 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-9f3308a4-8ff4-48ad-a465-803afd955615 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa0aaf9cdfcd49a9bd0578411989ac20', 'request_id': 'req-9f3308a4-8ff4-48ad-a465-803afd955615'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:40.663626 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-9f3308a4-8ff4-48ad-a465-803afd955615 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa0aaf9cdfcd49a9bd0578411989ac20', 'request_id': 'req-9f3308a4-8ff4-48ad-a465-803afd955615'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:40.669854 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-9f3308a4-8ff4-48ad-a465-803afd955615 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa0aaf9cdfcd49a9bd0578411989ac20', 'request_id': 'req-9f3308a4-8ff4-48ad-a465-803afd955615'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:40.683090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-9f3308a4-8ff4-48ad-a465-803afd955615 admin admin] The token cache is being invalidate because project fa0aaf9cdfcd49a9bd0578411989ac20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:03:40.683370 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-9f3308a4-8ff4-48ad-a465-803afd955615 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:40.683994 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-9f3308a4-8ff4-48ad-a465-803afd955615 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:40.685530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1222/2459] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:03:40 2022] DELETE /identity/v3/projects/fa0aaf9cdfcd49a9bd0578411989ac20 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:40.689585 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52956502-acee-4a8a-9060-1cd1d5b3ea77 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:40.699444 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52956502-acee-4a8a-9060-1cd1d5b3ea77 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:40.756593 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0055f-50a6-4590-92e9-8855d3b55513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:40.757460 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0055f-50a6-4590-92e9-8855d3b55513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:40.757744 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0055f-50a6-4590-92e9-8855d3b55513 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:40.758034 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0055f-50a6-4590-92e9-8855d3b55513 admin admin] PATH_INFO: `/v3/users/8413ba6618ad4db09b8c401cb19ebe9b` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:40.759559 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0055f-50a6-4590-92e9-8855d3b55513 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8413ba6618ad4db09b8c401cb19ebe9b)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:40.761503 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0055f-50a6-4590-92e9-8855d3b55513 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:40.807360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5cd0055f-50a6-4590-92e9-8855d3b55513 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8413ba6618ad4db09b8c401cb19ebe9b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:40.819270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5cd0055f-50a6-4590-92e9-8855d3b55513 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8413ba6618ad4db09b8c401cb19ebe9b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:40.827346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5cd0055f-50a6-4590-92e9-8855d3b55513 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8413ba6618ad4db09b8c401cb19ebe9b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:40.828190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5cd0055f-50a6-4590-92e9-8855d3b55513 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8413ba6618ad4db09b8c401cb19ebe9b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:40.828800 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5cd0055f-50a6-4590-92e9-8855d3b55513 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8413ba6618ad4db09b8c401cb19ebe9b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:40.841446 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1238/2460] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:03:40 2022] DELETE /identity/v3/users/8413ba6618ad4db09b8c401cb19ebe9b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:41.125489 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3308a4-8ff4-48ad-a465-803afd955615 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:41.127012 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-9f3308a4-8ff4-48ad-a465-803afd955615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:41.141160 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3308a4-8ff4-48ad-a465-803afd955615 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:41.200821 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf450cb-736a-4d6d-82d4-af8b04c1175a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:41.202257 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf450cb-736a-4d6d-82d4-af8b04c1175a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:41.202545 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf450cb-736a-4d6d-82d4-af8b04c1175a admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:41.202820 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf450cb-736a-4d6d-82d4-af8b04c1175a admin admin] PATH_INFO: `/v3/projects/30ae3fba388b4e13ad079a244e424587` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:41.204025 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf450cb-736a-4d6d-82d4-af8b04c1175a admin admin] RBAC: Authorizing `identity:delete_project(project_id=30ae3fba388b4e13ad079a244e424587)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:41.205978 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf450cb-736a-4d6d-82d4-af8b04c1175a admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:41.251631 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-7cf450cb-736a-4d6d-82d4-af8b04c1175a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30ae3fba388b4e13ad079a244e424587', 'request_id': 'req-7cf450cb-736a-4d6d-82d4-af8b04c1175a'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:41.257284 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-7cf450cb-736a-4d6d-82d4-af8b04c1175a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30ae3fba388b4e13ad079a244e424587', 'request_id': 'req-7cf450cb-736a-4d6d-82d4-af8b04c1175a'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:41.263731 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-7cf450cb-736a-4d6d-82d4-af8b04c1175a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30ae3fba388b4e13ad079a244e424587', 'request_id': 'req-7cf450cb-736a-4d6d-82d4-af8b04c1175a'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:41.276541 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-7cf450cb-736a-4d6d-82d4-af8b04c1175a admin admin] The token cache is being invalidate because project 30ae3fba388b4e13ad079a244e424587 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:03:41.276826 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-7cf450cb-736a-4d6d-82d4-af8b04c1175a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:41.277450 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-7cf450cb-736a-4d6d-82d4-af8b04c1175a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:41.279045 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1223/2461] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:03:41 2022] DELETE /identity/v3/projects/30ae3fba388b4e13ad079a244e424587 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:41.283830 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66eaa4f-eb68-49c5-9b29-9dfd7a0b244f None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:41.284330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66eaa4f-eb68-49c5-9b29-9dfd7a0b244f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:41.284433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66eaa4f-eb68-49c5-9b29-9dfd7a0b244f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:41.302511 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b66eaa4f-eb68-49c5-9b29-9dfd7a0b244f None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:41.303721 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b66eaa4f-eb68-49c5-9b29-9dfd7a0b244f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:41.351376 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:41.368038 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1239/2462] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:03:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:03:41.376876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63591b5a-93b7-4e42-9d77-7a2e7c2eadbf None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:41.377163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63591b5a-93b7-4e42-9d77-7a2e7c2eadbf None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:41.377442 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63591b5a-93b7-4e42-9d77-7a2e7c2eadbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:41.398275 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-63591b5a-93b7-4e42-9d77-7a2e7c2eadbf None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:41.399647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-63591b5a-93b7-4e42-9d77-7a2e7c2eadbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:41.407741 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1224/2463] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:03:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:03:41.412020 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66eaa4f-eb68-49c5-9b29-9dfd7a0b244f None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:41.422498 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66eaa4f-eb68-49c5-9b29-9dfd7a0b244f None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:41.427311 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49110149-a7f5-4d32-b548-853b9d18814f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:41.428064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49110149-a7f5-4d32-b548-853b9d18814f admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:41.428365 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49110149-a7f5-4d32-b548-853b9d18814f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:41.428665 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49110149-a7f5-4d32-b548-853b9d18814f admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:41.429536 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49110149-a7f5-4d32-b548-853b9d18814f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:03:41.429872 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49110149-a7f5-4d32-b548-853b9d18814f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:41.431514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49110149-a7f5-4d32-b548-853b9d18814f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:41.445372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1240/2464] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 15:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:03:41.448591 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63591b5a-93b7-4e42-9d77-7a2e7c2eadbf None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:41.458330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63591b5a-93b7-4e42-9d77-7a2e7c2eadbf None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:41.463054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce0d0d3-079c-419a-a09a-2d323b3cec37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:41.463705 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0d0d3-079c-419a-a09a-2d323b3cec37 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:41.463985 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0d0d3-079c-419a-a09a-2d323b3cec37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:41.464277 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0d0d3-079c-419a-a09a-2d323b3cec37 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:41.464954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce0d0d3-079c-419a-a09a-2d323b3cec37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:41.466912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce0d0d3-079c-419a-a09a-2d323b3cec37 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:41.494362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1225/2465] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:03:41 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:03:41.497406 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49110149-a7f5-4d32-b548-853b9d18814f admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:41.508026 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49110149-a7f5-4d32-b548-853b9d18814f admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:41.564143 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b8302-0bca-4a2b-935a-d063d9fc6754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:41.564809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b8302-0bca-4a2b-935a-d063d9fc6754 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:41.565098 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b8302-0bca-4a2b-935a-d063d9fc6754 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:41.565378 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b8302-0bca-4a2b-935a-d063d9fc6754 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:41.566089 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b8302-0bca-4a2b-935a-d063d9fc6754 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:41.568041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b8302-0bca-4a2b-935a-d063d9fc6754 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:41.587527 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1241/2466] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:03:41 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:03:41.591576 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce0d0d3-079c-419a-a09a-2d323b3cec37 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:41.602061 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce0d0d3-079c-419a-a09a-2d323b3cec37 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:41.606799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc165d8-859f-447f-8e04-897feaa619b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:41.607500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc165d8-859f-447f-8e04-897feaa619b3 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:41.608289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc165d8-859f-447f-8e04-897feaa619b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:41.608563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc165d8-859f-447f-8e04-897feaa619b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:41.609295 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc165d8-859f-447f-8e04-897feaa619b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:41.610961 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc165d8-859f-447f-8e04-897feaa619b3 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:41.621851 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1226/2467] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:03:41.626581 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b8302-0bca-4a2b-935a-d063d9fc6754 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:41.636788 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b8302-0bca-4a2b-935a-d063d9fc6754 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:41.641410 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97f897-ee34-4bfd-937c-45ea8df50e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:41.642098 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97f897-ee34-4bfd-937c-45ea8df50e35 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:41.642381 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97f897-ee34-4bfd-937c-45ea8df50e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:41.642654 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97f897-ee34-4bfd-937c-45ea8df50e35 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:41.643369 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97f897-ee34-4bfd-937c-45ea8df50e35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:41.644988 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97f897-ee34-4bfd-937c-45ea8df50e35 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:41.655104 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1242/2468] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:03:41.659280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc165d8-859f-447f-8e04-897feaa619b3 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:41.669458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc165d8-859f-447f-8e04-897feaa619b3 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:41.674594 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef8a32-37d3-4e10-a7af-4244ffb3e3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:41.675470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef8a32-37d3-4e10-a7af-4244ffb3e3be admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:41.675744 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef8a32-37d3-4e10-a7af-4244ffb3e3be admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:41.676008 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef8a32-37d3-4e10-a7af-4244ffb3e3be admin admin] PATH_INFO: `/v3/projects/71fb0659e24841419a39ba5d48d8f12f/users/c4c88c449c60465bac122aa3cfc31056/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:41.691018 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef8a32-37d3-4e10-a7af-4244ffb3e3be admin admin] RBAC: Authorizing `identity:create_grant(project_id=71fb0659e24841419a39ba5d48d8f12f, user_id=c4c88c449c60465bac122aa3cfc31056, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:41.693651 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef8a32-37d3-4e10-a7af-4244ffb3e3be admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:41.702557 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:41.712428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1227/2469] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:03:41 2022] PUT /identity/v3/projects/71fb0659e24841419a39ba5d48d8f12f/users/c4c88c449c60465bac122aa3cfc31056/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:41.713728 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce23670-58d4-42bf-b5f6-45e527ff3aab None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:41.714049 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce23670-58d4-42bf-b5f6-45e527ff3aab None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:41.714339 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce23670-58d4-42bf-b5f6-45e527ff3aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:41.737744 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-cce23670-58d4-42bf-b5f6-45e527ff3aab None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:41.739012 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-cce23670-58d4-42bf-b5f6-45e527ff3aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:41.800298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1243/2470] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:03:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:03:41.810065 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef8a32-37d3-4e10-a7af-4244ffb3e3be admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:41.811766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-56ef8a32-37d3-4e10-a7af-4244ffb3e3be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:41.869608 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef8a32-37d3-4e10-a7af-4244ffb3e3be admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:41.873269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29880a73-4b42-4ef3-a8fe-0e4a3b0957cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:41.873921 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29880a73-4b42-4ef3-a8fe-0e4a3b0957cb service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:41.874231 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29880a73-4b42-4ef3-a8fe-0e4a3b0957cb service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:41.874507 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29880a73-4b42-4ef3-a8fe-0e4a3b0957cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:41.883591 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29880a73-4b42-4ef3-a8fe-0e4a3b0957cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:41.885297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29880a73-4b42-4ef3-a8fe-0e4a3b0957cb service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:41.895017 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1228/2471] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:03:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:03:42.552502 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333b065-df05-499b-ab49-24ae93d598d2 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:42.553134 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333b065-df05-499b-ab49-24ae93d598d2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:42.553134 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333b065-df05-499b-ab49-24ae93d598d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:42.584032 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-6333b065-df05-499b-ab49-24ae93d598d2 None None] MFA Rules not processed for user `c4c88c449c60465bac122aa3cfc31056`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:42.585266 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-6333b065-df05-499b-ab49-24ae93d598d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:42.632840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:42.655456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1244/2472] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:03:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:03:42.660068 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29880a73-4b42-4ef3-a8fe-0e4a3b0957cb service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:42.669989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29880a73-4b42-4ef3-a8fe-0e4a3b0957cb service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:42.674789 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3caf69-135f-4833-a600-5e2e8cb8941a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:42.675446 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3caf69-135f-4833-a600-5e2e8cb8941a admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:42.675726 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3caf69-135f-4833-a600-5e2e8cb8941a admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:42.676010 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3caf69-135f-4833-a600-5e2e8cb8941a admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:42.676702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3caf69-135f-4833-a600-5e2e8cb8941a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:42.678659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3caf69-135f-4833-a600-5e2e8cb8941a admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:42.709640 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1229/2473] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:03:42 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:03:42.710691 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333b065-df05-499b-ab49-24ae93d598d2 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:42.721588 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333b065-df05-499b-ab49-24ae93d598d2 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:42.777837 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d46f2-d5d8-4b19-b29a-183e70bd4738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:42.778461 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d46f2-d5d8-4b19-b29a-183e70bd4738 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:42.778749 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d46f2-d5d8-4b19-b29a-183e70bd4738 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:42.779030 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d46f2-d5d8-4b19-b29a-183e70bd4738 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:42.779734 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d46f2-d5d8-4b19-b29a-183e70bd4738 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:42.781681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d46f2-d5d8-4b19-b29a-183e70bd4738 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:42.800756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1245/2474] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:03:42 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:03:42.805137 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3caf69-135f-4833-a600-5e2e8cb8941a admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:42.816656 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3caf69-135f-4833-a600-5e2e8cb8941a admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:42.821294 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09997a-ff13-4d1e-9376-70bf6f49066c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:42.821954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09997a-ff13-4d1e-9376-70bf6f49066c admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:42.822283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09997a-ff13-4d1e-9376-70bf6f49066c admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:42.822561 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09997a-ff13-4d1e-9376-70bf6f49066c admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:42.823311 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09997a-ff13-4d1e-9376-70bf6f49066c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:42.824908 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09997a-ff13-4d1e-9376-70bf6f49066c admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:42.836053 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1230/2475] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:03:42.840510 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d46f2-d5d8-4b19-b29a-183e70bd4738 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:42.851144 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d46f2-d5d8-4b19-b29a-183e70bd4738 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:42.855927 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38e181-50b6-4aa5-9002-5d874fb8bde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:42.856802 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38e181-50b6-4aa5-9002-5d874fb8bde3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:42.857085 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38e181-50b6-4aa5-9002-5d874fb8bde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:42.857360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38e181-50b6-4aa5-9002-5d874fb8bde3 admin admin] PATH_INFO: `/v3/projects/3537dd7522384099af154e2bfe6ece59/users/d0b02267cce34431bbf837c7f926e7f4/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:42.872247 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38e181-50b6-4aa5-9002-5d874fb8bde3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3537dd7522384099af154e2bfe6ece59, user_id=d0b02267cce34431bbf837c7f926e7f4, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:42.875513 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38e181-50b6-4aa5-9002-5d874fb8bde3 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:42.890227 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1246/2476] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:03:42 2022] PUT /identity/v3/projects/3537dd7522384099af154e2bfe6ece59/users/d0b02267cce34431bbf837c7f926e7f4/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:42.894629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09997a-ff13-4d1e-9376-70bf6f49066c admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:42.909426 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09997a-ff13-4d1e-9376-70bf6f49066c admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:42.966095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e66bb-668c-4c65-b4f2-45d149334cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:42.966763 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e66bb-668c-4c65-b4f2-45d149334cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:42.967047 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e66bb-668c-4c65-b4f2-45d149334cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:42.967325 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e66bb-668c-4c65-b4f2-45d149334cd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:42.968041 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039e66bb-668c-4c65-b4f2-45d149334cd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:42.969641 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039e66bb-668c-4c65-b4f2-45d149334cd1 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:42.979752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1231/2477] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:03:42.984563 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38e181-50b6-4aa5-9002-5d874fb8bde3 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:42.994831 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38e181-50b6-4aa5-9002-5d874fb8bde3 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:43.000054 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705cf8b-1b49-453a-8490-817d7e266b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:43.000932 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705cf8b-1b49-453a-8490-817d7e266b6d admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:43.001219 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705cf8b-1b49-453a-8490-817d7e266b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:43.001501 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705cf8b-1b49-453a-8490-817d7e266b6d admin admin] PATH_INFO: `/v3/projects/3537dd7522384099af154e2bfe6ece59/users/d0b02267cce34431bbf837c7f926e7f4/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:43.003531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5705cf8b-1b49-453a-8490-817d7e266b6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3537dd7522384099af154e2bfe6ece59, user_id=d0b02267cce34431bbf837c7f926e7f4, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:43.006396 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5705cf8b-1b49-453a-8490-817d7e266b6d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:43.016909 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1247/2478] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:03:42 2022] PUT /identity/v3/projects/3537dd7522384099af154e2bfe6ece59/users/d0b02267cce34431bbf837c7f926e7f4/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:03:43.681208 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad73fa3-322d-4d9c-8d8b-0c712c9d8936 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:43.681782 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad73fa3-322d-4d9c-8d8b-0c712c9d8936 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:43.681882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad73fa3-322d-4d9c-8d8b-0c712c9d8936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:43.711319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-7ad73fa3-322d-4d9c-8d8b-0c712c9d8936 None None] MFA Rules not processed for user `d0b02267cce34431bbf837c7f926e7f4`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:43.712573 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-7ad73fa3-322d-4d9c-8d8b-0c712c9d8936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:43.763792 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:43.783857 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1232/2479] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:03:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:03:43.794966 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705cf8b-1b49-453a-8490-817d7e266b6d admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:43.796758 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5705cf8b-1b49-453a-8490-817d7e266b6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:43.857352 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705cf8b-1b49-453a-8490-817d7e266b6d admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:43.861030 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4997d6-5b27-4b53-8a5f-8a8e44d6984c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:43.861686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4997d6-5b27-4b53-8a5f-8a8e44d6984c service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:43.861969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4997d6-5b27-4b53-8a5f-8a8e44d6984c service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:43.862286 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4997d6-5b27-4b53-8a5f-8a8e44d6984c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:43.922027 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:43.927179 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4997d6-5b27-4b53-8a5f-8a8e44d6984c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:43.929018 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4997d6-5b27-4b53-8a5f-8a8e44d6984c service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:43.939233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1248/2480] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:03:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:03:45.294820 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad73fa3-322d-4d9c-8d8b-0c712c9d8936 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:45.296256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-7ad73fa3-322d-4d9c-8d8b-0c712c9d8936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:45.308240 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad73fa3-322d-4d9c-8d8b-0c712c9d8936 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:45.311804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa40e0f-3985-4c20-bac6-97cf575943dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:45.312441 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa40e0f-3985-4c20-bac6-97cf575943dc service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:45.312723 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa40e0f-3985-4c20-bac6-97cf575943dc service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:45.313005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa40e0f-3985-4c20-bac6-97cf575943dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:45.322213 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa40e0f-3985-4c20-bac6-97cf575943dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:45.323645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa40e0f-3985-4c20-bac6-97cf575943dc service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:45.333730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1233/2481] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:03:45 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:03:53.935075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:55.329517 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=10 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:58.457463 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4997d6-5b27-4b53-8a5f-8a8e44d6984c service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:58.527607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4997d6-5b27-4b53-8a5f-8a8e44d6984c service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:58.532424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982af9b-329d-463b-9807-80483ab7c067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:58.533310 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982af9b-329d-463b-9807-80483ab7c067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:58.533595 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982af9b-329d-463b-9807-80483ab7c067 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:58.533870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982af9b-329d-463b-9807-80483ab7c067 admin admin] PATH_INFO: `/v3/users/c4c88c449c60465bac122aa3cfc31056` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:58.535159 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982af9b-329d-463b-9807-80483ab7c067 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4c88c449c60465bac122aa3cfc31056)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:58.537125 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982af9b-329d-463b-9807-80483ab7c067 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:58.588474 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-7982af9b-329d-463b-9807-80483ab7c067 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4c88c449c60465bac122aa3cfc31056'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:58.601860 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-7982af9b-329d-463b-9807-80483ab7c067 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4c88c449c60465bac122aa3cfc31056'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:58.611127 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-7982af9b-329d-463b-9807-80483ab7c067 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4c88c449c60465bac122aa3cfc31056'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:58.611880 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-7982af9b-329d-463b-9807-80483ab7c067 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4c88c449c60465bac122aa3cfc31056'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:58.612617 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-7982af9b-329d-463b-9807-80483ab7c067 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4c88c449c60465bac122aa3cfc31056'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:58.627475 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1249/2482] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:03:58 2022] DELETE /identity/v3/users/c4c88c449c60465bac122aa3cfc31056 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:58.810458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa40e0f-3985-4c20-bac6-97cf575943dc service cinder] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:58.825171 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa40e0f-3985-4c20-bac6-97cf575943dc service cinder] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:58.890773 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c872257b-ca1f-419e-a72f-531f494dc2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:58.891548 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872257b-ca1f-419e-a72f-531f494dc2cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:58.891825 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872257b-ca1f-419e-a72f-531f494dc2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:58.892102 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872257b-ca1f-419e-a72f-531f494dc2cf admin admin] PATH_INFO: `/v3/projects/71fb0659e24841419a39ba5d48d8f12f` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:58.893313 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c872257b-ca1f-419e-a72f-531f494dc2cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=71fb0659e24841419a39ba5d48d8f12f)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:58.895281 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c872257b-ca1f-419e-a72f-531f494dc2cf admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:58.944707 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c872257b-ca1f-419e-a72f-531f494dc2cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71fb0659e24841419a39ba5d48d8f12f', 'request_id': 'req-c872257b-ca1f-419e-a72f-531f494dc2cf'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:58.950259 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c872257b-ca1f-419e-a72f-531f494dc2cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71fb0659e24841419a39ba5d48d8f12f', 'request_id': 'req-c872257b-ca1f-419e-a72f-531f494dc2cf'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:58.956759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c872257b-ca1f-419e-a72f-531f494dc2cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71fb0659e24841419a39ba5d48d8f12f', 'request_id': 'req-c872257b-ca1f-419e-a72f-531f494dc2cf'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:58.969798 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c872257b-ca1f-419e-a72f-531f494dc2cf admin admin] The token cache is being invalidate because project 71fb0659e24841419a39ba5d48d8f12f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:03:58.970106 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c872257b-ca1f-419e-a72f-531f494dc2cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:58.970765 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c872257b-ca1f-419e-a72f-531f494dc2cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:58.972454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1234/2483] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:03:58 2022] DELETE /identity/v3/projects/71fb0659e24841419a39ba5d48d8f12f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:58.976681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982af9b-329d-463b-9807-80483ab7c067 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:58.978151 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7982af9b-329d-463b-9807-80483ab7c067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:58.992371 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982af9b-329d-463b-9807-80483ab7c067 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:59.055597 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d477a7-7cbc-43c4-8ea7-e824b5db62fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:59.056483 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d477a7-7cbc-43c4-8ea7-e824b5db62fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:59.056773 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d477a7-7cbc-43c4-8ea7-e824b5db62fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:59.057050 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d477a7-7cbc-43c4-8ea7-e824b5db62fb admin admin] PATH_INFO: `/v3/users/d0b02267cce34431bbf837c7f926e7f4` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:59.058300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d477a7-7cbc-43c4-8ea7-e824b5db62fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0b02267cce34431bbf837c7f926e7f4)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:59.060249 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d477a7-7cbc-43c4-8ea7-e824b5db62fb admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:59.092763 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:59.122584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-14d477a7-7cbc-43c4-8ea7-e824b5db62fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0b02267cce34431bbf837c7f926e7f4'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:59.135358 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-14d477a7-7cbc-43c4-8ea7-e824b5db62fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0b02267cce34431bbf837c7f926e7f4'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:59.142477 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-14d477a7-7cbc-43c4-8ea7-e824b5db62fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0b02267cce34431bbf837c7f926e7f4'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:59.143032 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-14d477a7-7cbc-43c4-8ea7-e824b5db62fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0b02267cce34431bbf837c7f926e7f4'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:59.143629 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-14d477a7-7cbc-43c4-8ea7-e824b5db62fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0b02267cce34431bbf837c7f926e7f4'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:59.156669 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1250/2484] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:03:58 2022] DELETE /identity/v3/users/d0b02267cce34431bbf837c7f926e7f4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:59.333618 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c872257b-ca1f-419e-a72f-531f494dc2cf admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:59.335109 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-c872257b-ca1f-419e-a72f-531f494dc2cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:59.348598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c872257b-ca1f-419e-a72f-531f494dc2cf admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:59.417113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d124c-0b28-4d1b-994f-6c25ddc863e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:59.418179 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d124c-0b28-4d1b-994f-6c25ddc863e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:59.418689 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d124c-0b28-4d1b-994f-6c25ddc863e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:59.419095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d124c-0b28-4d1b-994f-6c25ddc863e7 admin admin] PATH_INFO: `/v3/projects/3537dd7522384099af154e2bfe6ece59` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:59.421593 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d124c-0b28-4d1b-994f-6c25ddc863e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3537dd7522384099af154e2bfe6ece59)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:59.424122 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d124c-0b28-4d1b-994f-6c25ddc863e7 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:59.475245 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:03:59.490153 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-603d124c-0b28-4d1b-994f-6c25ddc863e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3537dd7522384099af154e2bfe6ece59', 'request_id': 'req-603d124c-0b28-4d1b-994f-6c25ddc863e7'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:59.496185 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-603d124c-0b28-4d1b-994f-6c25ddc863e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3537dd7522384099af154e2bfe6ece59', 'request_id': 'req-603d124c-0b28-4d1b-994f-6c25ddc863e7'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:59.503761 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-603d124c-0b28-4d1b-994f-6c25ddc863e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3537dd7522384099af154e2bfe6ece59', 'request_id': 'req-603d124c-0b28-4d1b-994f-6c25ddc863e7'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:59.519619 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-603d124c-0b28-4d1b-994f-6c25ddc863e7 admin admin] The token cache is being invalidate because project 3537dd7522384099af154e2bfe6ece59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:03:59.519904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-603d124c-0b28-4d1b-994f-6c25ddc863e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:59.520551 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-603d124c-0b28-4d1b-994f-6c25ddc863e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:03:59.522248 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1235/2485] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:03:59 2022] DELETE /identity/v3/projects/3537dd7522384099af154e2bfe6ece59 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:59.527295 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f178dbd-249d-491f-9631-c35e1619a826 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:59.527853 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f178dbd-249d-491f-9631-c35e1619a826 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:59.527853 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f178dbd-249d-491f-9631-c35e1619a826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:59.547584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-4f178dbd-249d-491f-9631-c35e1619a826 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:59.548806 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4f178dbd-249d-491f-9631-c35e1619a826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:59.612283 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1251/2486] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:03:59.622816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7126a-df28-47bd-81f4-a5bf68273aa3 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:59.623154 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7126a-df28-47bd-81f4-a5bf68273aa3 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:59.623482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7126a-df28-47bd-81f4-a5bf68273aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:59.643830 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-32c7126a-df28-47bd-81f4-a5bf68273aa3 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:03:59.645066 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-32c7126a-df28-47bd-81f4-a5bf68273aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:03:59.653161 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1236/2487] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:03:59.658005 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f178dbd-249d-491f-9631-c35e1619a826 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:59.667999 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f178dbd-249d-491f-9631-c35e1619a826 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:59.672577 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327fc35-c2ba-4907-b4ca-9c68f21f3c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:59.673314 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327fc35-c2ba-4907-b4ca-9c68f21f3c82 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:59.673599 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327fc35-c2ba-4907-b4ca-9c68f21f3c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:59.673876 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327fc35-c2ba-4907-b4ca-9c68f21f3c82 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:59.674644 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8327fc35-c2ba-4907-b4ca-9c68f21f3c82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:03:59.674950 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8327fc35-c2ba-4907-b4ca-9c68f21f3c82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:59.676595 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8327fc35-c2ba-4907-b4ca-9c68f21f3c82 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:59.690463 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1252/2488] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 15:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:03:59.695324 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7126a-df28-47bd-81f4-a5bf68273aa3 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:59.705917 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7126a-df28-47bd-81f4-a5bf68273aa3 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:59.711387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f635630-d895-4fb7-aa6c-c615a30a5b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:59.712074 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f635630-d895-4fb7-aa6c-c615a30a5b28 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:59.712358 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f635630-d895-4fb7-aa6c-c615a30a5b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:59.712636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f635630-d895-4fb7-aa6c-c615a30a5b28 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:59.713341 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f635630-d895-4fb7-aa6c-c615a30a5b28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:59.715419 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f635630-d895-4fb7-aa6c-c615a30a5b28 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:59.749002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1237/2489] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:03:59 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:03:59.750313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327fc35-c2ba-4907-b4ca-9c68f21f3c82 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:59.761106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327fc35-c2ba-4907-b4ca-9c68f21f3c82 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:59.822194 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8eaaa0-30d3-45d7-8bc9-6160fb81b18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:59.822894 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8eaaa0-30d3-45d7-8bc9-6160fb81b18f admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:59.823249 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8eaaa0-30d3-45d7-8bc9-6160fb81b18f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:59.823587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8eaaa0-30d3-45d7-8bc9-6160fb81b18f admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:59.824321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8eaaa0-30d3-45d7-8bc9-6160fb81b18f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:59.826244 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8eaaa0-30d3-45d7-8bc9-6160fb81b18f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:59.845993 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1253/2490] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:03:59 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:03:59.850127 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f635630-d895-4fb7-aa6c-c615a30a5b28 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:59.860355 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f635630-d895-4fb7-aa6c-c615a30a5b28 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:59.864949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ab364-d035-4145-b710-c2c3c1a25c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:59.865601 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ab364-d035-4145-b710-c2c3c1a25c82 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:59.865880 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ab364-d035-4145-b710-c2c3c1a25c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:59.866183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ab364-d035-4145-b710-c2c3c1a25c82 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:59.867002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78ab364-d035-4145-b710-c2c3c1a25c82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:59.868512 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78ab364-d035-4145-b710-c2c3c1a25c82 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:59.879749 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1238/2491] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:03:59.883791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8eaaa0-30d3-45d7-8bc9-6160fb81b18f admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:59.894573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8eaaa0-30d3-45d7-8bc9-6160fb81b18f admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:59.899179 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169e987-6440-45d3-9e0e-b46372b34b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:59.899832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169e987-6440-45d3-9e0e-b46372b34b4c admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:59.900114 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169e987-6440-45d3-9e0e-b46372b34b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:59.900384 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169e987-6440-45d3-9e0e-b46372b34b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:59.901434 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a169e987-6440-45d3-9e0e-b46372b34b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:59.903071 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a169e987-6440-45d3-9e0e-b46372b34b4c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:59.915328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1254/2492] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:03:59.919716 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ab364-d035-4145-b710-c2c3c1a25c82 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:03:59.932571 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ab364-d035-4145-b710-c2c3c1a25c82 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:03:59.937854 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cbb645-076b-4b12-8640-fc019f67c2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:03:59.938789 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cbb645-076b-4b12-8640-fc019f67c2d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:59.939071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cbb645-076b-4b12-8640-fc019f67c2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:59.939346 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cbb645-076b-4b12-8640-fc019f67c2d2 admin admin] PATH_INFO: `/v3/projects/68a4a529c33f40e6a4f8728233110b68/users/315f015b401c40f4bd1f14f4e63240ac/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:59.955244 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cbb645-076b-4b12-8640-fc019f67c2d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68a4a529c33f40e6a4f8728233110b68, user_id=315f015b401c40f4bd1f14f4e63240ac, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:03:59.957944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cbb645-076b-4b12-8640-fc019f67c2d2 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:03:59.971802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1239/2493] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:03:59 2022] PUT /identity/v3/projects/68a4a529c33f40e6a4f8728233110b68/users/315f015b401c40f4bd1f14f4e63240ac/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:03:59.982992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7945774-fd98-4ee8-a13e-b9a70fb64cf8 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:03:59.983271 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7945774-fd98-4ee8-a13e-b9a70fb64cf8 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:03:59.983556 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7945774-fd98-4ee8-a13e-b9a70fb64cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:03:59.990755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:00.015726 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-e7945774-fd98-4ee8-a13e-b9a70fb64cf8 None None] MFA Rules not processed for user `315f015b401c40f4bd1f14f4e63240ac`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:00.017023 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e7945774-fd98-4ee8-a13e-b9a70fb64cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:00.091030 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1255/2494] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:03:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:00.096037 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cbb645-076b-4b12-8640-fc019f67c2d2 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:00.172153 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cbb645-076b-4b12-8640-fc019f67c2d2 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:00.176870 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617372f-162b-4f15-958a-b1190b34e991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:00.177529 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617372f-162b-4f15-958a-b1190b34e991 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:00.177810 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617372f-162b-4f15-958a-b1190b34e991 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:00.178110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617372f-162b-4f15-958a-b1190b34e991 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:00.178802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617372f-162b-4f15-958a-b1190b34e991 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:00.180699 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617372f-162b-4f15-958a-b1190b34e991 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:00.206472 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1240/2495] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:04:00 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:00.210753 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7945774-fd98-4ee8-a13e-b9a70fb64cf8 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:00.222870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7945774-fd98-4ee8-a13e-b9a70fb64cf8 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:00.279298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72057620-663e-4201-86a5-e7ae52f6cdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:00.279975 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72057620-663e-4201-86a5-e7ae52f6cdc5 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:00.280253 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72057620-663e-4201-86a5-e7ae52f6cdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:00.280530 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72057620-663e-4201-86a5-e7ae52f6cdc5 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:00.281208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72057620-663e-4201-86a5-e7ae52f6cdc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:00.283175 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72057620-663e-4201-86a5-e7ae52f6cdc5 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:00.303979 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1256/2496] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:04:00 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:00.308015 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617372f-162b-4f15-958a-b1190b34e991 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:00.320472 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617372f-162b-4f15-958a-b1190b34e991 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:00.325569 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9a7a2-1e06-40fd-8740-85b76f2c4af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:00.326345 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9a7a2-1e06-40fd-8740-85b76f2c4af2 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:00.326752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9a7a2-1e06-40fd-8740-85b76f2c4af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:00.327094 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9a7a2-1e06-40fd-8740-85b76f2c4af2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:00.327889 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9a7a2-1e06-40fd-8740-85b76f2c4af2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:00.329572 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9a7a2-1e06-40fd-8740-85b76f2c4af2 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:00.341054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1241/2497] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:00.345279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72057620-663e-4201-86a5-e7ae52f6cdc5 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:00.355433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72057620-663e-4201-86a5-e7ae52f6cdc5 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:00.360623 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f41d19-f055-4c07-8914-e7b4d5b7c77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:00.361502 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f41d19-f055-4c07-8914-e7b4d5b7c77d admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:00.361781 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f41d19-f055-4c07-8914-e7b4d5b7c77d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:00.362084 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f41d19-f055-4c07-8914-e7b4d5b7c77d admin admin] PATH_INFO: `/v3/projects/c600d873db644b07abb8f3185e9c11c4/users/182ae671d68147149f5851027c46152e/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:00.378482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f41d19-f055-4c07-8914-e7b4d5b7c77d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c600d873db644b07abb8f3185e9c11c4, user_id=182ae671d68147149f5851027c46152e, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:00.381192 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f41d19-f055-4c07-8914-e7b4d5b7c77d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:00.397306 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1257/2498] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:00 2022] PUT /identity/v3/projects/c600d873db644b07abb8f3185e9c11c4/users/182ae671d68147149f5851027c46152e/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:00.401290 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9a7a2-1e06-40fd-8740-85b76f2c4af2 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:00.411686 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9a7a2-1e06-40fd-8740-85b76f2c4af2 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:00.441435 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:00.469389 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb618af6-6c2c-4dab-b71b-0a592dc5eb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:00.470082 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb618af6-6c2c-4dab-b71b-0a592dc5eb66 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:00.470366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb618af6-6c2c-4dab-b71b-0a592dc5eb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:00.470639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb618af6-6c2c-4dab-b71b-0a592dc5eb66 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:00.471353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb618af6-6c2c-4dab-b71b-0a592dc5eb66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:00.472946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb618af6-6c2c-4dab-b71b-0a592dc5eb66 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:00.482650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1242/2499] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:00.487566 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f41d19-f055-4c07-8914-e7b4d5b7c77d admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:00.497959 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f41d19-f055-4c07-8914-e7b4d5b7c77d admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:00.503359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da70990-f5b4-415c-9b94-65f73b854fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:00.504242 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da70990-f5b4-415c-9b94-65f73b854fc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:00.504524 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da70990-f5b4-415c-9b94-65f73b854fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:00.504802 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da70990-f5b4-415c-9b94-65f73b854fc4 admin admin] PATH_INFO: `/v3/projects/c600d873db644b07abb8f3185e9c11c4/users/182ae671d68147149f5851027c46152e/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:00.506853 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da70990-f5b4-415c-9b94-65f73b854fc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c600d873db644b07abb8f3185e9c11c4, user_id=182ae671d68147149f5851027c46152e, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:00.509806 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da70990-f5b4-415c-9b94-65f73b854fc4 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:00.520314 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1258/2500] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:00 2022] PUT /identity/v3/projects/c600d873db644b07abb8f3185e9c11c4/users/182ae671d68147149f5851027c46152e/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:00.531331 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc9b22-2e30-4807-bdee-fd7e7d1aafc7 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:00.531618 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc9b22-2e30-4807-bdee-fd7e7d1aafc7 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:00.531901 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc9b22-2e30-4807-bdee-fd7e7d1aafc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:00.559320 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-c7fc9b22-2e30-4807-bdee-fd7e7d1aafc7 None None] MFA Rules not processed for user `182ae671d68147149f5851027c46152e`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:00.560478 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-c7fc9b22-2e30-4807-bdee-fd7e7d1aafc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:00.643121 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1243/2501] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:00.653889 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da70990-f5b4-415c-9b94-65f73b854fc4 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:00.656138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9da70990-f5b4-415c-9b94-65f73b854fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:00.711086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:00.718056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da70990-f5b4-415c-9b94-65f73b854fc4 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:00.722411 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf70813-52d7-473f-a622-d4f9dd59e997 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:00.723089 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf70813-52d7-473f-a622-d4f9dd59e997 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:00.723390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf70813-52d7-473f-a622-d4f9dd59e997 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:00.723664 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf70813-52d7-473f-a622-d4f9dd59e997 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:00.782867 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf70813-52d7-473f-a622-d4f9dd59e997 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:00.784368 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf70813-52d7-473f-a622-d4f9dd59e997 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:00.793998 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1259/2502] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:05.124256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc9b22-2e30-4807-bdee-fd7e7d1aafc7 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:05.125703 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-c7fc9b22-2e30-4807-bdee-fd7e7d1aafc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:05.138579 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc9b22-2e30-4807-bdee-fd7e7d1aafc7 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:05.142803 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc8c92-276e-4807-a5e2-71d5b895ef40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:05.143451 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc8c92-276e-4807-a5e2-71d5b895ef40 service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:05.143731 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc8c92-276e-4807-a5e2-71d5b895ef40 service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:05.144006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc8c92-276e-4807-a5e2-71d5b895ef40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:05.153703 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdc8c92-276e-4807-a5e2-71d5b895ef40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:05.155172 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdc8c92-276e-4807-a5e2-71d5b895ef40 service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:05.165229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1244/2503] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:10.061085 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf70813-52d7-473f-a622-d4f9dd59e997 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:10.126415 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf70813-52d7-473f-a622-d4f9dd59e997 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:10.132214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9fbb8-af5a-4f38-9ba7-396c97a2b113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:10.133234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9fbb8-af5a-4f38-9ba7-396c97a2b113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:10.133510 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9fbb8-af5a-4f38-9ba7-396c97a2b113 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:10.133750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9fbb8-af5a-4f38-9ba7-396c97a2b113 admin admin] PATH_INFO: `/v3/users/315f015b401c40f4bd1f14f4e63240ac` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:10.135457 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba9fbb8-af5a-4f38-9ba7-396c97a2b113 admin admin] RBAC: Authorizing `identity:delete_user(user_id=315f015b401c40f4bd1f14f4e63240ac)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:10.137429 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba9fbb8-af5a-4f38-9ba7-396c97a2b113 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:10.187888 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3ba9fbb8-af5a-4f38-9ba7-396c97a2b113 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '315f015b401c40f4bd1f14f4e63240ac'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:10.200164 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3ba9fbb8-af5a-4f38-9ba7-396c97a2b113 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '315f015b401c40f4bd1f14f4e63240ac'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:10.213248 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3ba9fbb8-af5a-4f38-9ba7-396c97a2b113 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '315f015b401c40f4bd1f14f4e63240ac'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:10.213894 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3ba9fbb8-af5a-4f38-9ba7-396c97a2b113 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '315f015b401c40f4bd1f14f4e63240ac'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:10.214619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3ba9fbb8-af5a-4f38-9ba7-396c97a2b113 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '315f015b401c40f4bd1f14f4e63240ac'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:10.230765 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1260/2504] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:04:10 2022] DELETE /identity/v3/users/315f015b401c40f4bd1f14f4e63240ac => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:10.235666 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173476ee-b840-447d-98b8-b6c524043270 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:10.235666 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173476ee-b840-447d-98b8-b6c524043270 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:10.236248 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173476ee-b840-447d-98b8-b6c524043270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:10.254446 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-173476ee-b840-447d-98b8-b6c524043270 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:10.255676 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-173476ee-b840-447d-98b8-b6c524043270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:10.329276 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1245/2505] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:10.338109 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9fbb8-af5a-4f38-9ba7-396c97a2b113 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:10.339609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3ba9fbb8-af5a-4f38-9ba7-396c97a2b113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:10.363427 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:10.407399 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9fbb8-af5a-4f38-9ba7-396c97a2b113 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:10.411322 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfb4192-0af8-486b-bf80-1cf5cf85b345 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:10.411978 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb4192-0af8-486b-bf80-1cf5cf85b345 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:10.412266 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb4192-0af8-486b-bf80-1cf5cf85b345 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:10.412541 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb4192-0af8-486b-bf80-1cf5cf85b345 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:10.426363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfb4192-0af8-486b-bf80-1cf5cf85b345 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:10.427851 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfb4192-0af8-486b-bf80-1cf5cf85b345 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:10.438605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1261/2506] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:10.788172 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173476ee-b840-447d-98b8-b6c524043270 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:10.789953 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-173476ee-b840-447d-98b8-b6c524043270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:10.813614 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173476ee-b840-447d-98b8-b6c524043270 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:10.820644 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cac5d8-3415-485b-b904-e136d1560a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:10.821536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cac5d8-3415-485b-b904-e136d1560a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:10.821818 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cac5d8-3415-485b-b904-e136d1560a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:10.822274 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cac5d8-3415-485b-b904-e136d1560a4d admin admin] PATH_INFO: `/v3/projects/68a4a529c33f40e6a4f8728233110b68` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:10.823682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cac5d8-3415-485b-b904-e136d1560a4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=68a4a529c33f40e6a4f8728233110b68)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:10.827166 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cac5d8-3415-485b-b904-e136d1560a4d admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:10.890001 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:10.905302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-52cac5d8-3415-485b-b904-e136d1560a4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68a4a529c33f40e6a4f8728233110b68', 'request_id': 'req-52cac5d8-3415-485b-b904-e136d1560a4d'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:10.913925 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-52cac5d8-3415-485b-b904-e136d1560a4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68a4a529c33f40e6a4f8728233110b68', 'request_id': 'req-52cac5d8-3415-485b-b904-e136d1560a4d'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:10.921935 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-52cac5d8-3415-485b-b904-e136d1560a4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68a4a529c33f40e6a4f8728233110b68', 'request_id': 'req-52cac5d8-3415-485b-b904-e136d1560a4d'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:10.936757 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-52cac5d8-3415-485b-b904-e136d1560a4d admin admin] The token cache is being invalidate because project 68a4a529c33f40e6a4f8728233110b68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:04:10.937065 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-52cac5d8-3415-485b-b904-e136d1560a4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:10.938428 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-52cac5d8-3415-485b-b904-e136d1560a4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:10.940472 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1246/2507] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:04:10 2022] DELETE /identity/v3/projects/68a4a529c33f40e6a4f8728233110b68 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:10.945755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfb4192-0af8-486b-bf80-1cf5cf85b345 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:10.959092 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfb4192-0af8-486b-bf80-1cf5cf85b345 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:11.030359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d8f93-45d4-4198-9501-4a5734f2c839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:11.031371 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d8f93-45d4-4198-9501-4a5734f2c839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:11.031650 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d8f93-45d4-4198-9501-4a5734f2c839 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:11.031922 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d8f93-45d4-4198-9501-4a5734f2c839 admin admin] PATH_INFO: `/v3/users/182ae671d68147149f5851027c46152e` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:11.033133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d8f93-45d4-4198-9501-4a5734f2c839 admin admin] RBAC: Authorizing `identity:delete_user(user_id=182ae671d68147149f5851027c46152e)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:11.035137 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d8f93-45d4-4198-9501-4a5734f2c839 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:11.088446 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-968d8f93-45d4-4198-9501-4a5734f2c839 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '182ae671d68147149f5851027c46152e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:11.101838 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-968d8f93-45d4-4198-9501-4a5734f2c839 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '182ae671d68147149f5851027c46152e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:11.115245 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-968d8f93-45d4-4198-9501-4a5734f2c839 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '182ae671d68147149f5851027c46152e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:11.115967 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-968d8f93-45d4-4198-9501-4a5734f2c839 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '182ae671d68147149f5851027c46152e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:11.116636 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-968d8f93-45d4-4198-9501-4a5734f2c839 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '182ae671d68147149f5851027c46152e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:11.141235 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1262/2508] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:04:10 2022] DELETE /identity/v3/users/182ae671d68147149f5851027c46152e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:04:11.484550 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cac5d8-3415-485b-b904-e136d1560a4d admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:11.486549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-52cac5d8-3415-485b-b904-e136d1560a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:11.500880 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cac5d8-3415-485b-b904-e136d1560a4d admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:11.571445 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf7de5-3416-4aee-9428-8c77b3d20b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:11.572329 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf7de5-3416-4aee-9428-8c77b3d20b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:11.572602 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf7de5-3416-4aee-9428-8c77b3d20b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:11.572877 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf7de5-3416-4aee-9428-8c77b3d20b35 admin admin] PATH_INFO: `/v3/projects/c600d873db644b07abb8f3185e9c11c4` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:11.574140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdf7de5-3416-4aee-9428-8c77b3d20b35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c600d873db644b07abb8f3185e9c11c4)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:11.576141 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdf7de5-3416-4aee-9428-8c77b3d20b35 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:11.642165 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-1cdf7de5-3416-4aee-9428-8c77b3d20b35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c600d873db644b07abb8f3185e9c11c4', 'request_id': 'req-1cdf7de5-3416-4aee-9428-8c77b3d20b35'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:11.647996 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-1cdf7de5-3416-4aee-9428-8c77b3d20b35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c600d873db644b07abb8f3185e9c11c4', 'request_id': 'req-1cdf7de5-3416-4aee-9428-8c77b3d20b35'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:11.655811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-1cdf7de5-3416-4aee-9428-8c77b3d20b35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c600d873db644b07abb8f3185e9c11c4', 'request_id': 'req-1cdf7de5-3416-4aee-9428-8c77b3d20b35'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:11.672871 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-1cdf7de5-3416-4aee-9428-8c77b3d20b35 admin admin] The token cache is being invalidate because project c600d873db644b07abb8f3185e9c11c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:04:11.673511 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-1cdf7de5-3416-4aee-9428-8c77b3d20b35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:11.673909 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-1cdf7de5-3416-4aee-9428-8c77b3d20b35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:11.675689 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1247/2509] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:04:11 2022] DELETE /identity/v3/projects/c600d873db644b07abb8f3185e9c11c4 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:11.683101 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bb848-2c5a-4a15-b337-05ed71d9d610 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:11.683433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bb848-2c5a-4a15-b337-05ed71d9d610 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:11.683675 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bb848-2c5a-4a15-b337-05ed71d9d610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:11.705033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-060bb848-2c5a-4a15-b337-05ed71d9d610 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:11.706427 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-060bb848-2c5a-4a15-b337-05ed71d9d610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:11.754926 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:11.786725 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1263/2510] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:04:11.795966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ac7d2-6026-421e-aefb-ca1d211f0cdf None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:11.796249 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ac7d2-6026-421e-aefb-ca1d211f0cdf None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:11.796520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ac7d2-6026-421e-aefb-ca1d211f0cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:11.815517 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-2c8ac7d2-6026-421e-aefb-ca1d211f0cdf None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:11.816720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-2c8ac7d2-6026-421e-aefb-ca1d211f0cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:11.824725 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1248/2511] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:11.829570 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060bb848-2c5a-4a15-b337-05ed71d9d610 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:11.839356 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060bb848-2c5a-4a15-b337-05ed71d9d610 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:11.843974 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c39ca8e-5efc-4954-a821-c524a5b4ce7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:11.844635 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39ca8e-5efc-4954-a821-c524a5b4ce7f admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:11.844940 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39ca8e-5efc-4954-a821-c524a5b4ce7f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:11.845221 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39ca8e-5efc-4954-a821-c524a5b4ce7f admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:11.845945 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c39ca8e-5efc-4954-a821-c524a5b4ce7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:04:11.846294 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c39ca8e-5efc-4954-a821-c524a5b4ce7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:11.847908 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c39ca8e-5efc-4954-a821-c524a5b4ce7f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:11.859649 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1264/2512] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 15:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:04:11.864076 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ac7d2-6026-421e-aefb-ca1d211f0cdf None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:11.878691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ac7d2-6026-421e-aefb-ca1d211f0cdf None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:11.883888 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef6f31-b98b-48fa-baf6-e7e14da7390f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:11.884824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef6f31-b98b-48fa-baf6-e7e14da7390f admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:11.885098 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef6f31-b98b-48fa-baf6-e7e14da7390f admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:11.885366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef6f31-b98b-48fa-baf6-e7e14da7390f admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:11.886079 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef6f31-b98b-48fa-baf6-e7e14da7390f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:11.887981 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef6f31-b98b-48fa-baf6-e7e14da7390f admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:11.919114 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1249/2513] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:04:11 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:11.920625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c39ca8e-5efc-4954-a821-c524a5b4ce7f admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:11.931190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c39ca8e-5efc-4954-a821-c524a5b4ce7f admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:11.990311 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d155082-629a-45d2-aad0-a97d961477b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:11.991076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d155082-629a-45d2-aad0-a97d961477b5 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:11.991459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d155082-629a-45d2-aad0-a97d961477b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:11.991744 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d155082-629a-45d2-aad0-a97d961477b5 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:11.992442 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d155082-629a-45d2-aad0-a97d961477b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:11.994581 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d155082-629a-45d2-aad0-a97d961477b5 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:12.040256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1265/2514] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:04:11 2022] POST /identity/v3/users => generated 292 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:12.049305 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef6f31-b98b-48fa-baf6-e7e14da7390f admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:12.059961 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef6f31-b98b-48fa-baf6-e7e14da7390f admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:12.068612 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b63ca-81ce-47ba-a9b4-98514593b62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:12.069309 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b63ca-81ce-47ba-a9b4-98514593b62f admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:12.069619 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b63ca-81ce-47ba-a9b4-98514593b62f admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:12.069900 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b63ca-81ce-47ba-a9b4-98514593b62f admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:12.070683 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b63ca-81ce-47ba-a9b4-98514593b62f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:12.072628 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b63ca-81ce-47ba-a9b4-98514593b62f admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:12.091188 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1250/2515] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:04:12.092558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d155082-629a-45d2-aad0-a97d961477b5 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:12.113809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d155082-629a-45d2-aad0-a97d961477b5 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:12.118860 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943199a-2bb9-4537-a4ae-a8caa6a85957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:12.119546 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943199a-2bb9-4537-a4ae-a8caa6a85957 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:12.120007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943199a-2bb9-4537-a4ae-a8caa6a85957 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:12.120304 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943199a-2bb9-4537-a4ae-a8caa6a85957 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:12.121089 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943199a-2bb9-4537-a4ae-a8caa6a85957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:12.122998 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943199a-2bb9-4537-a4ae-a8caa6a85957 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:12.136201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1266/2516] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:12.141176 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b63ca-81ce-47ba-a9b4-98514593b62f admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:12.156586 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b63ca-81ce-47ba-a9b4-98514593b62f admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:12.163726 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27450b3b-790c-4146-ac21-941a322d0487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:12.164923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27450b3b-790c-4146-ac21-941a322d0487 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:12.165370 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27450b3b-790c-4146-ac21-941a322d0487 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:12.166185 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27450b3b-790c-4146-ac21-941a322d0487 admin admin] PATH_INFO: `/v3/projects/f35f870ccda840bbabd78dd2c7d8cc11/users/d62ad2c992a8474b95cee5b94088b8a2/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:12.183741 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:12.192607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27450b3b-790c-4146-ac21-941a322d0487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f35f870ccda840bbabd78dd2c7d8cc11, user_id=d62ad2c992a8474b95cee5b94088b8a2, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:12.197351 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27450b3b-790c-4146-ac21-941a322d0487 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:12.219074 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1251/2517] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:12 2022] PUT /identity/v3/projects/f35f870ccda840bbabd78dd2c7d8cc11/users/d62ad2c992a8474b95cee5b94088b8a2/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:12.230623 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbaed9e-8108-4321-8666-e9cd891d897e None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:12.231159 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbaed9e-8108-4321-8666-e9cd891d897e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:12.231452 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbaed9e-8108-4321-8666-e9cd891d897e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:12.268160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-9dbaed9e-8108-4321-8666-e9cd891d897e None None] MFA Rules not processed for user `d62ad2c992a8474b95cee5b94088b8a2`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:12.269707 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9dbaed9e-8108-4321-8666-e9cd891d897e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:12.390181 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1267/2518] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:12.400267 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27450b3b-790c-4146-ac21-941a322d0487 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:12.402809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-27450b3b-790c-4146-ac21-941a322d0487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:12.485068 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27450b3b-790c-4146-ac21-941a322d0487 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:12.489642 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc77fe-aaef-46ac-a31c-ae246baca855 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:12.490415 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc77fe-aaef-46ac-a31c-ae246baca855 service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:12.490769 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc77fe-aaef-46ac-a31c-ae246baca855 service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:12.491191 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc77fe-aaef-46ac-a31c-ae246baca855 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:12.501272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc77fe-aaef-46ac-a31c-ae246baca855 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:12.502951 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc77fe-aaef-46ac-a31c-ae246baca855 service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:12.516500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1252/2519] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:04:19.750923 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbaed9e-8108-4321-8666-e9cd891d897e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:19.753657 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9dbaed9e-8108-4321-8666-e9cd891d897e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:19.794745 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:19.811729 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbaed9e-8108-4321-8666-e9cd891d897e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:19.815367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77add6-d3ec-4346-a04c-558f366b04c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:19.816020 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77add6-d3ec-4346-a04c-558f366b04c3 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:19.816303 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77add6-d3ec-4346-a04c-558f366b04c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:19.816581 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77add6-d3ec-4346-a04c-558f366b04c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:19.818397 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2a77add6-d3ec-4346-a04c-558f366b04c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:19.891974 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a77add6-d3ec-4346-a04c-558f366b04c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:19.893459 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a77add6-d3ec-4346-a04c-558f366b04c3 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:19.906915 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1268/2520] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:21.112372 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc77fe-aaef-46ac-a31c-ae246baca855 service cinder] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:21.124941 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc77fe-aaef-46ac-a31c-ae246baca855 service cinder] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:21.129618 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2a98e-2ec6-4f91-94bf-4d509014174e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:21.130631 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2a98e-2ec6-4f91-94bf-4d509014174e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:21.130979 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2a98e-2ec6-4f91-94bf-4d509014174e admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:21.131322 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2a98e-2ec6-4f91-94bf-4d509014174e admin admin] PATH_INFO: `/v3/users/d62ad2c992a8474b95cee5b94088b8a2` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:21.132683 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2a98e-2ec6-4f91-94bf-4d509014174e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d62ad2c992a8474b95cee5b94088b8a2)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:21.135019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2a98e-2ec6-4f91-94bf-4d509014174e admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:21.189591 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-23a2a98e-2ec6-4f91-94bf-4d509014174e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd62ad2c992a8474b95cee5b94088b8a2'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:21.203535 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-23a2a98e-2ec6-4f91-94bf-4d509014174e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd62ad2c992a8474b95cee5b94088b8a2'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:21.212083 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-23a2a98e-2ec6-4f91-94bf-4d509014174e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd62ad2c992a8474b95cee5b94088b8a2'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:21.212826 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-23a2a98e-2ec6-4f91-94bf-4d509014174e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd62ad2c992a8474b95cee5b94088b8a2'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:21.213511 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-23a2a98e-2ec6-4f91-94bf-4d509014174e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd62ad2c992a8474b95cee5b94088b8a2'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:21.227640 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1253/2521] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:04:21 2022] DELETE /identity/v3/users/d62ad2c992a8474b95cee5b94088b8a2 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:21.232359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0af88-c507-43b7-aa94-81aee5b6433a None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:21.233816 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0af88-c507-43b7-aa94-81aee5b6433a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:21.233816 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0af88-c507-43b7-aa94-81aee5b6433a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:21.251430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-1af0af88-c507-43b7-aa94-81aee5b6433a None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:21.252664 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1af0af88-c507-43b7-aa94-81aee5b6433a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:21.325151 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1269/2522] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:21.335178 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2a98e-2ec6-4f91-94bf-4d509014174e admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:21.337199 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-23a2a98e-2ec6-4f91-94bf-4d509014174e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:21.400859 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2a98e-2ec6-4f91-94bf-4d509014174e admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:21.404722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b834e4-3069-459c-997b-8dc7150f64a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:21.405386 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b834e4-3069-459c-997b-8dc7150f64a6 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:21.405673 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b834e4-3069-459c-997b-8dc7150f64a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:21.405953 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b834e4-3069-459c-997b-8dc7150f64a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:21.415861 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b834e4-3069-459c-997b-8dc7150f64a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:21.417370 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b834e4-3069-459c-997b-8dc7150f64a6 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:21.428359 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1254/2523] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:04:21.729862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0af88-c507-43b7-aa94-81aee5b6433a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:21.731347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1af0af88-c507-43b7-aa94-81aee5b6433a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:21.753755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0af88-c507-43b7-aa94-81aee5b6433a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:21.761121 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeffec0-8b83-42bc-9b5c-0c3874ad3bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:21.762307 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeffec0-8b83-42bc-9b5c-0c3874ad3bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:21.762819 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeffec0-8b83-42bc-9b5c-0c3874ad3bea admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:21.763260 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeffec0-8b83-42bc-9b5c-0c3874ad3bea admin admin] PATH_INFO: `/v3/projects/f35f870ccda840bbabd78dd2c7d8cc11` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:21.765167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeffec0-8b83-42bc-9b5c-0c3874ad3bea admin admin] RBAC: Authorizing `identity:delete_project(project_id=f35f870ccda840bbabd78dd2c7d8cc11)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:21.768169 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeffec0-8b83-42bc-9b5c-0c3874ad3bea admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:21.827294 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5aeffec0-8b83-42bc-9b5c-0c3874ad3bea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f35f870ccda840bbabd78dd2c7d8cc11', 'request_id': 'req-5aeffec0-8b83-42bc-9b5c-0c3874ad3bea'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:21.833664 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5aeffec0-8b83-42bc-9b5c-0c3874ad3bea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f35f870ccda840bbabd78dd2c7d8cc11', 'request_id': 'req-5aeffec0-8b83-42bc-9b5c-0c3874ad3bea'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:21.840241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5aeffec0-8b83-42bc-9b5c-0c3874ad3bea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f35f870ccda840bbabd78dd2c7d8cc11', 'request_id': 'req-5aeffec0-8b83-42bc-9b5c-0c3874ad3bea'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:21.854572 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5aeffec0-8b83-42bc-9b5c-0c3874ad3bea admin admin] The token cache is being invalidate because project f35f870ccda840bbabd78dd2c7d8cc11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:04:21.855007 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5aeffec0-8b83-42bc-9b5c-0c3874ad3bea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:21.855878 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5aeffec0-8b83-42bc-9b5c-0c3874ad3bea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:21.858454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1270/2524] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:04:21 2022] DELETE /identity/v3/projects/f35f870ccda840bbabd78dd2c7d8cc11 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:21.863307 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27682ba-d590-418e-829f-e5dd2a1e34b1 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:21.863681 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27682ba-d590-418e-829f-e5dd2a1e34b1 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:21.863962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27682ba-d590-418e-829f-e5dd2a1e34b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:21.877472 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:21.886388 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-c27682ba-d590-418e-829f-e5dd2a1e34b1 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:21.888701 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-c27682ba-d590-418e-829f-e5dd2a1e34b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:21.973606 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1255/2525] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:21.981888 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3b2af-b60b-4237-a61f-0ec3c478c885 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:21.982286 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3b2af-b60b-4237-a61f-0ec3c478c885 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:21.982672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3b2af-b60b-4237-a61f-0ec3c478c885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:22.002660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-fed3b2af-b60b-4237-a61f-0ec3c478c885 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:22.004201 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-fed3b2af-b60b-4237-a61f-0ec3c478c885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:22.013891 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1271/2526] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:22.018397 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27682ba-d590-418e-829f-e5dd2a1e34b1 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:22.030036 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27682ba-d590-418e-829f-e5dd2a1e34b1 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:22.035427 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65cdea-5f16-46dd-9a75-75c37350af3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:22.036092 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65cdea-5f16-46dd-9a75-75c37350af3a admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:22.036374 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65cdea-5f16-46dd-9a75-75c37350af3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:22.036649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65cdea-5f16-46dd-9a75-75c37350af3a admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:22.037381 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65cdea-5f16-46dd-9a75-75c37350af3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:04:22.037686 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65cdea-5f16-46dd-9a75-75c37350af3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:22.039368 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65cdea-5f16-46dd-9a75-75c37350af3a admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:22.063324 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1256/2527] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 15:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:04:22.064480 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3b2af-b60b-4237-a61f-0ec3c478c885 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:22.073461 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:22.078387 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3b2af-b60b-4237-a61f-0ec3c478c885 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:22.085846 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e50d5-b016-4bc9-9166-44512f979f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:22.086862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e50d5-b016-4bc9-9166-44512f979f6b admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:22.087506 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e50d5-b016-4bc9-9166-44512f979f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:22.088148 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e50d5-b016-4bc9-9166-44512f979f6b admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:22.089941 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e50d5-b016-4bc9-9166-44512f979f6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:22.092039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e50d5-b016-4bc9-9166-44512f979f6b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:22.141155 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1272/2528] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:04:22 2022] POST /identity/v3/projects => generated 375 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:22.147747 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65cdea-5f16-46dd-9a75-75c37350af3a admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:22.161462 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65cdea-5f16-46dd-9a75-75c37350af3a admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:22.237776 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c687fa-6287-4eb5-9275-99acaf562acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:22.238535 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c687fa-6287-4eb5-9275-99acaf562acc admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:22.238857 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c687fa-6287-4eb5-9275-99acaf562acc admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:22.239179 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c687fa-6287-4eb5-9275-99acaf562acc admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:22.240056 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c687fa-6287-4eb5-9275-99acaf562acc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:22.242232 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c687fa-6287-4eb5-9275-99acaf562acc admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:22.266052 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1257/2529] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:04:22 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:22.270477 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e50d5-b016-4bc9-9166-44512f979f6b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:22.283582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e50d5-b016-4bc9-9166-44512f979f6b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:22.288982 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fcd4e-5c8e-45bb-a438-fe6e9e4becf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:22.289780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fcd4e-5c8e-45bb-a438-fe6e9e4becf8 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:22.290159 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fcd4e-5c8e-45bb-a438-fe6e9e4becf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:22.290509 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fcd4e-5c8e-45bb-a438-fe6e9e4becf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:22.291323 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0fcd4e-5c8e-45bb-a438-fe6e9e4becf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:22.293076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0fcd4e-5c8e-45bb-a438-fe6e9e4becf8 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:22.308668 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1273/2530] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:22.309868 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c687fa-6287-4eb5-9275-99acaf562acc admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:22.322462 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c687fa-6287-4eb5-9275-99acaf562acc admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:22.326299 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137f984-3bfa-43db-a88b-ac68d7fc265e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:22.327104 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137f984-3bfa-43db-a88b-ac68d7fc265e admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:22.327480 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137f984-3bfa-43db-a88b-ac68d7fc265e admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:22.327879 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137f984-3bfa-43db-a88b-ac68d7fc265e admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:22.328741 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137f984-3bfa-43db-a88b-ac68d7fc265e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:22.330505 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137f984-3bfa-43db-a88b-ac68d7fc265e admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:22.343691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1258/2531] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:22.347911 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fcd4e-5c8e-45bb-a438-fe6e9e4becf8 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:22.359601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fcd4e-5c8e-45bb-a438-fe6e9e4becf8 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:22.364362 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d9294-59f4-4d3a-87d1-50ce444e4520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:22.365268 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d9294-59f4-4d3a-87d1-50ce444e4520 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:22.365534 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d9294-59f4-4d3a-87d1-50ce444e4520 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:22.365941 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d9294-59f4-4d3a-87d1-50ce444e4520 admin admin] PATH_INFO: `/v3/projects/750ba9582a504688b8beb0dae10297e9/users/2195dbeb77804451911c7af202cadcc5/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:22.383046 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d9294-59f4-4d3a-87d1-50ce444e4520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=750ba9582a504688b8beb0dae10297e9, user_id=2195dbeb77804451911c7af202cadcc5, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:22.386288 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d9294-59f4-4d3a-87d1-50ce444e4520 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:22.413897 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1274/2532] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:22 2022] PUT /identity/v3/projects/750ba9582a504688b8beb0dae10297e9/users/2195dbeb77804451911c7af202cadcc5/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:04:22.415424 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b413b8-20d9-459d-92f9-29cbd50906f8 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:22.415706 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b413b8-20d9-459d-92f9-29cbd50906f8 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:22.415972 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b413b8-20d9-459d-92f9-29cbd50906f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:22.444203 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-79b413b8-20d9-459d-92f9-29cbd50906f8 None None] MFA Rules not processed for user `2195dbeb77804451911c7af202cadcc5`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:22.445426 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-79b413b8-20d9-459d-92f9-29cbd50906f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:22.512096 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1259/2533] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:04:22.522479 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d9294-59f4-4d3a-87d1-50ce444e4520 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:22.524194 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-798d9294-59f4-4d3a-87d1-50ce444e4520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:22.588565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d9294-59f4-4d3a-87d1-50ce444e4520 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:22.592873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e115bbf-f56b-409b-96c0-669005ca9b76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:22.593532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e115bbf-f56b-409b-96c0-669005ca9b76 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:22.593821 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e115bbf-f56b-409b-96c0-669005ca9b76 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:22.594132 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e115bbf-f56b-409b-96c0-669005ca9b76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:22.603553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e115bbf-f56b-409b-96c0-669005ca9b76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:22.605031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e115bbf-f56b-409b-96c0-669005ca9b76 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:22.615774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1275/2534] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:23.770878 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b413b8-20d9-459d-92f9-29cbd50906f8 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:23.772312 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-79b413b8-20d9-459d-92f9-29cbd50906f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:23.777445 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:23.843096 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b413b8-20d9-459d-92f9-29cbd50906f8 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:23.847542 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d5df8-6761-46ce-8b87-4cf19318f573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:23.848414 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d5df8-6761-46ce-8b87-4cf19318f573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:23.848696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d5df8-6761-46ce-8b87-4cf19318f573 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:23.848980 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d5df8-6761-46ce-8b87-4cf19318f573 admin admin] PATH_INFO: `/v3/users/e60a5e751f22450c8e366c85b37af1f6` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:23.850494 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d5df8-6761-46ce-8b87-4cf19318f573 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e60a5e751f22450c8e366c85b37af1f6)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:23.852430 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d5df8-6761-46ce-8b87-4cf19318f573 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:23.901298 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b61d5df8-6761-46ce-8b87-4cf19318f573 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e60a5e751f22450c8e366c85b37af1f6'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:23.913689 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b61d5df8-6761-46ce-8b87-4cf19318f573 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e60a5e751f22450c8e366c85b37af1f6'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:23.921054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b61d5df8-6761-46ce-8b87-4cf19318f573 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e60a5e751f22450c8e366c85b37af1f6'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:23.921681 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b61d5df8-6761-46ce-8b87-4cf19318f573 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e60a5e751f22450c8e366c85b37af1f6'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:23.922312 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b61d5df8-6761-46ce-8b87-4cf19318f573 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e60a5e751f22450c8e366c85b37af1f6'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:23.934658 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1260/2535] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:04:23 2022] DELETE /identity/v3/users/e60a5e751f22450c8e366c85b37af1f6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:24.128057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e115bbf-f56b-409b-96c0-669005ca9b76 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:24.140312 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e115bbf-f56b-409b-96c0-669005ca9b76 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:24.210487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041403e4-e3d7-42ba-9d81-2496e829792e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:24.211356 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041403e4-e3d7-42ba-9d81-2496e829792e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:24.211638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041403e4-e3d7-42ba-9d81-2496e829792e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:24.211914 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041403e4-e3d7-42ba-9d81-2496e829792e admin admin] PATH_INFO: `/v3/projects/9dafcd0df8524413b4fe81a0931323b1` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:24.213123 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041403e4-e3d7-42ba-9d81-2496e829792e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dafcd0df8524413b4fe81a0931323b1)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:24.215085 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041403e4-e3d7-42ba-9d81-2496e829792e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:24.265934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-041403e4-e3d7-42ba-9d81-2496e829792e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dafcd0df8524413b4fe81a0931323b1', 'request_id': 'req-041403e4-e3d7-42ba-9d81-2496e829792e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:24.267951 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:24.272190 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-041403e4-e3d7-42ba-9d81-2496e829792e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dafcd0df8524413b4fe81a0931323b1', 'request_id': 'req-041403e4-e3d7-42ba-9d81-2496e829792e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:24.279368 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-041403e4-e3d7-42ba-9d81-2496e829792e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dafcd0df8524413b4fe81a0931323b1', 'request_id': 'req-041403e4-e3d7-42ba-9d81-2496e829792e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:24.294209 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-041403e4-e3d7-42ba-9d81-2496e829792e admin admin] The token cache is being invalidate because project 9dafcd0df8524413b4fe81a0931323b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:04:24.294505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-041403e4-e3d7-42ba-9d81-2496e829792e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:24.295145 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-041403e4-e3d7-42ba-9d81-2496e829792e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:24.296769 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1276/2536] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:04:24 2022] DELETE /identity/v3/projects/9dafcd0df8524413b4fe81a0931323b1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:24.301910 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49caa8-7a28-4442-bf5b-3f3496e90c7d None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:24.302262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49caa8-7a28-4442-bf5b-3f3496e90c7d None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:24.302561 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49caa8-7a28-4442-bf5b-3f3496e90c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:24.320822 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-2a49caa8-7a28-4442-bf5b-3f3496e90c7d None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:24.322189 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-2a49caa8-7a28-4442-bf5b-3f3496e90c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:24.393321 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1261/2537] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:24.402089 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd2098-97f8-4ee5-bedb-96686e282bbe None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:24.402494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd2098-97f8-4ee5-bedb-96686e282bbe None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:24.402803 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd2098-97f8-4ee5-bedb-96686e282bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:24.423136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b3bd2098-97f8-4ee5-bedb-96686e282bbe None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:24.424354 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b3bd2098-97f8-4ee5-bedb-96686e282bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:24.441016 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1277/2538] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:04:24.443108 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49caa8-7a28-4442-bf5b-3f3496e90c7d None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:24.455630 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49caa8-7a28-4442-bf5b-3f3496e90c7d None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:24.460942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b237e8-9d05-4da4-8372-a2b73a45da5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:24.461616 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b237e8-9d05-4da4-8372-a2b73a45da5f admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:24.461895 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b237e8-9d05-4da4-8372-a2b73a45da5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:24.462261 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b237e8-9d05-4da4-8372-a2b73a45da5f admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:24.463063 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b237e8-9d05-4da4-8372-a2b73a45da5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:04:24.463374 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b237e8-9d05-4da4-8372-a2b73a45da5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:24.465806 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b237e8-9d05-4da4-8372-a2b73a45da5f admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:24.482489 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1262/2539] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 15:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:04:24.487696 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd2098-97f8-4ee5-bedb-96686e282bbe None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:24.499674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd2098-97f8-4ee5-bedb-96686e282bbe None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:24.504328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52bb28-9703-4cb3-914c-3cf38f9661e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:24.504993 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52bb28-9703-4cb3-914c-3cf38f9661e2 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:24.505282 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52bb28-9703-4cb3-914c-3cf38f9661e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:24.505547 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52bb28-9703-4cb3-914c-3cf38f9661e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:24.506263 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52bb28-9703-4cb3-914c-3cf38f9661e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:24.508177 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52bb28-9703-4cb3-914c-3cf38f9661e2 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:24.541608 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1278/2540] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:04:24 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:04:24.545445 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b237e8-9d05-4da4-8372-a2b73a45da5f admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:24.556358 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b237e8-9d05-4da4-8372-a2b73a45da5f admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:24.561949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:24.645541 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d96f3-56a2-404a-a0e9-9d3701a9fbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:24.646246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d96f3-56a2-404a-a0e9-9d3701a9fbb0 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:24.646526 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d96f3-56a2-404a-a0e9-9d3701a9fbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:24.646802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d96f3-56a2-404a-a0e9-9d3701a9fbb0 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:24.647497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9d96f3-56a2-404a-a0e9-9d3701a9fbb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:24.649637 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9d96f3-56a2-404a-a0e9-9d3701a9fbb0 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:24.670363 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1263/2541] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:04:24 2022] POST /identity/v3/users => generated 290 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:04:24.674796 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52bb28-9703-4cb3-914c-3cf38f9661e2 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:24.687218 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52bb28-9703-4cb3-914c-3cf38f9661e2 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:24.692693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacda860-f216-42a4-b30d-7112b1d6788c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:24.693499 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacda860-f216-42a4-b30d-7112b1d6788c admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:24.693852 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacda860-f216-42a4-b30d-7112b1d6788c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:24.694236 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacda860-f216-42a4-b30d-7112b1d6788c admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:24.695040 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacda860-f216-42a4-b30d-7112b1d6788c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:24.696767 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacda860-f216-42a4-b30d-7112b1d6788c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:24.708620 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1279/2542] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:24.712804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d96f3-56a2-404a-a0e9-9d3701a9fbb0 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:24.724032 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d96f3-56a2-404a-a0e9-9d3701a9fbb0 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:24.729052 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1057e-15ad-422c-8a45-8a4e18516ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:24.729718 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1057e-15ad-422c-8a45-8a4e18516ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:24.729997 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1057e-15ad-422c-8a45-8a4e18516ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:24.730304 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1057e-15ad-422c-8a45-8a4e18516ff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:24.731027 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1057e-15ad-422c-8a45-8a4e18516ff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:24.732680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1057e-15ad-422c-8a45-8a4e18516ff1 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:24.748130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1264/2543] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:24.749955 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacda860-f216-42a4-b30d-7112b1d6788c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:24.761489 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacda860-f216-42a4-b30d-7112b1d6788c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:24.766885 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d591c6-408d-4123-911d-887f1d6d8a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:24.767909 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d591c6-408d-4123-911d-887f1d6d8a43 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:24.768196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d591c6-408d-4123-911d-887f1d6d8a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:24.768473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d591c6-408d-4123-911d-887f1d6d8a43 admin admin] PATH_INFO: `/v3/projects/a71ae27fc182483884e94bc7435a99df/users/008eb91ab3fe4c4e9e6d8771d42c2bc3/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:24.783507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d591c6-408d-4123-911d-887f1d6d8a43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a71ae27fc182483884e94bc7435a99df, user_id=008eb91ab3fe4c4e9e6d8771d42c2bc3, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:24.786296 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d591c6-408d-4123-911d-887f1d6d8a43 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:24.800732 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1280/2544] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:24 2022] PUT /identity/v3/projects/a71ae27fc182483884e94bc7435a99df/users/008eb91ab3fe4c4e9e6d8771d42c2bc3/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:04:24.810566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5bdfc-cafe-462d-a9f0-f3c3af7482dd None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:24.811639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5bdfc-cafe-462d-a9f0-f3c3af7482dd None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:24.811753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5bdfc-cafe-462d-a9f0-f3c3af7482dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:24.840279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-f3b5bdfc-cafe-462d-a9f0-f3c3af7482dd None None] MFA Rules not processed for user `008eb91ab3fe4c4e9e6d8771d42c2bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:24.841382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f3b5bdfc-cafe-462d-a9f0-f3c3af7482dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:24.907793 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1265/2545] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:24.912725 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d591c6-408d-4123-911d-887f1d6d8a43 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:24.978505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d591c6-408d-4123-911d-887f1d6d8a43 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:24.984879 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ed06e-7038-4c13-aff4-6b55fee692c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:24.985595 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ed06e-7038-4c13-aff4-6b55fee692c9 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:24.985936 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ed06e-7038-4c13-aff4-6b55fee692c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:24.986312 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ed06e-7038-4c13-aff4-6b55fee692c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:24.987039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ed06e-7038-4c13-aff4-6b55fee692c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:24.989349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ed06e-7038-4c13-aff4-6b55fee692c9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:25.021512 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1281/2546] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:04:24 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:25.026657 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5bdfc-cafe-462d-a9f0-f3c3af7482dd None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:25.038960 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5bdfc-cafe-462d-a9f0-f3c3af7482dd None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:25.100134 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c1308-5e62-49c8-ac7b-efee85cbf6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:25.100724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c1308-5e62-49c8-ac7b-efee85cbf6e9 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:25.101009 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c1308-5e62-49c8-ac7b-efee85cbf6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:25.101288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c1308-5e62-49c8-ac7b-efee85cbf6e9 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:25.101990 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c1308-5e62-49c8-ac7b-efee85cbf6e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:25.104014 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c1308-5e62-49c8-ac7b-efee85cbf6e9 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:25.125578 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1266/2547] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:04:25 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:04:25.129739 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ed06e-7038-4c13-aff4-6b55fee692c9 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:25.140609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ed06e-7038-4c13-aff4-6b55fee692c9 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:25.145292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb469f-310a-46d2-a5a4-f6e91989b4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:25.145950 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb469f-310a-46d2-a5a4-f6e91989b4eb admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:25.146272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb469f-310a-46d2-a5a4-f6e91989b4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:25.146553 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb469f-310a-46d2-a5a4-f6e91989b4eb admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:25.147278 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb469f-310a-46d2-a5a4-f6e91989b4eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:25.148892 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb469f-310a-46d2-a5a4-f6e91989b4eb admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:25.159512 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1282/2548] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:25.163573 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c1308-5e62-49c8-ac7b-efee85cbf6e9 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:25.174050 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c1308-5e62-49c8-ac7b-efee85cbf6e9 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:25.178730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc203b8c-4373-4172-ba0d-4edde11bfbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:25.179600 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc203b8c-4373-4172-ba0d-4edde11bfbbf admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:25.179881 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc203b8c-4373-4172-ba0d-4edde11bfbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:25.180156 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc203b8c-4373-4172-ba0d-4edde11bfbbf admin admin] PATH_INFO: `/v3/projects/b6522666621246c3a37f4b660ce4dbf0/users/fd992e99ca7745afa886fd06a0f60fc3/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:25.182407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:25.197412 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc203b8c-4373-4172-ba0d-4edde11bfbbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6522666621246c3a37f4b660ce4dbf0, user_id=fd992e99ca7745afa886fd06a0f60fc3, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:25.200110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc203b8c-4373-4172-ba0d-4edde11bfbbf admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:25.212761 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1267/2549] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:25 2022] PUT /identity/v3/projects/b6522666621246c3a37f4b660ce4dbf0/users/fd992e99ca7745afa886fd06a0f60fc3/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:25.216704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb469f-310a-46d2-a5a4-f6e91989b4eb admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:25.227646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb469f-310a-46d2-a5a4-f6e91989b4eb admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:25.274656 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:25.286290 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df718583-5559-40cb-a0a7-8d76bb40f48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:25.286958 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df718583-5559-40cb-a0a7-8d76bb40f48c admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:25.287249 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df718583-5559-40cb-a0a7-8d76bb40f48c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:25.287526 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df718583-5559-40cb-a0a7-8d76bb40f48c admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:25.288247 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df718583-5559-40cb-a0a7-8d76bb40f48c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:25.289859 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df718583-5559-40cb-a0a7-8d76bb40f48c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:25.300265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1283/2550] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:25.304484 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc203b8c-4373-4172-ba0d-4edde11bfbbf admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:25.315521 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc203b8c-4373-4172-ba0d-4edde11bfbbf admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:25.320107 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52d242-95a1-4208-a615-4136a71c714e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:25.320983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52d242-95a1-4208-a615-4136a71c714e admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:25.321265 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52d242-95a1-4208-a615-4136a71c714e admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:25.321538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52d242-95a1-4208-a615-4136a71c714e admin admin] PATH_INFO: `/v3/projects/b6522666621246c3a37f4b660ce4dbf0/users/fd992e99ca7745afa886fd06a0f60fc3/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:25.323575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52d242-95a1-4208-a615-4136a71c714e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6522666621246c3a37f4b660ce4dbf0, user_id=fd992e99ca7745afa886fd06a0f60fc3, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:25.326271 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52d242-95a1-4208-a615-4136a71c714e admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:25.336559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1268/2551] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:25 2022] PUT /identity/v3/projects/b6522666621246c3a37f4b660ce4dbf0/users/fd992e99ca7745afa886fd06a0f60fc3/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:25.348061 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df6dd0-f0d4-4e77-9ffa-d23c9f657a8a None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:25.348347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df6dd0-f0d4-4e77-9ffa-d23c9f657a8a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:25.348622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df6dd0-f0d4-4e77-9ffa-d23c9f657a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:25.377233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-58df6dd0-f0d4-4e77-9ffa-d23c9f657a8a None None] MFA Rules not processed for user `fd992e99ca7745afa886fd06a0f60fc3`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:25.378480 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-58df6dd0-f0d4-4e77-9ffa-d23c9f657a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:25.445927 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1284/2552] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:25 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:25.458234 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52d242-95a1-4208-a615-4136a71c714e admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:25.459949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-9d52d242-95a1-4208-a615-4136a71c714e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:25.518560 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52d242-95a1-4208-a615-4136a71c714e admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:25.522315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5006e-ebc0-47b2-9596-4d5ec16502d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:25.522960 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5006e-ebc0-47b2-9596-4d5ec16502d1 service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:25.523243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5006e-ebc0-47b2-9596-4d5ec16502d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:25.523517 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5006e-ebc0-47b2-9596-4d5ec16502d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:25.536759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e5006e-ebc0-47b2-9596-4d5ec16502d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:25.538405 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e5006e-ebc0-47b2-9596-4d5ec16502d1 service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:25.548957 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1269/2553] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:25 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:25.771162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df6dd0-f0d4-4e77-9ffa-d23c9f657a8a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:25.772983 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-58df6dd0-f0d4-4e77-9ffa-d23c9f657a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:25.847439 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df6dd0-f0d4-4e77-9ffa-d23c9f657a8a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:25.852639 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5c7d0-5203-443c-b413-06d0c43e12ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:25.853585 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5c7d0-5203-443c-b413-06d0c43e12ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:25.853942 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5c7d0-5203-443c-b413-06d0c43e12ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:25.854314 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5c7d0-5203-443c-b413-06d0c43e12ea admin admin] PATH_INFO: `/v3/users/2195dbeb77804451911c7af202cadcc5` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:25.855705 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5c7d0-5203-443c-b413-06d0c43e12ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=2195dbeb77804451911c7af202cadcc5)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:25.857836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5c7d0-5203-443c-b413-06d0c43e12ea admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:25.887372 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5006e-ebc0-47b2-9596-4d5ec16502d1 service cinder] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:25.905775 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5006e-ebc0-47b2-9596-4d5ec16502d1 service cinder] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:25.912256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e38cb00-08a5-4bc9-aa88-3be7804119b1 tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd992e99ca7745afa886fd06a0f60fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6522666621246c3a37f4b660ce4dbf0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:25.912930 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38cb00-08a5-4bc9-aa88-3be7804119b1 tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:25.913222 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38cb00-08a5-4bc9-aa88-3be7804119b1 tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:25.913657 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38cb00-08a5-4bc9-aa88-3be7804119b1 tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:25.914433 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e38cb00-08a5-4bc9-aa88-3be7804119b1 tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:04:25.914742 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e38cb00-08a5-4bc9-aa88-3be7804119b1 tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:25.915516 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-53d5c7d0-5203-443c-b413-06d0c43e12ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2195dbeb77804451911c7af202cadcc5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:25.916350 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e38cb00-08a5-4bc9-aa88-3be7804119b1 tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:25.928000 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-53d5c7d0-5203-443c-b413-06d0c43e12ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2195dbeb77804451911c7af202cadcc5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:25.928525 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1270/2554] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 15:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:04:25.933085 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e38cb00-08a5-4bc9-aa88-3be7804119b1 tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:25.936820 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-53d5c7d0-5203-443c-b413-06d0c43e12ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2195dbeb77804451911c7af202cadcc5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:25.937464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-53d5c7d0-5203-443c-b413-06d0c43e12ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2195dbeb77804451911c7af202cadcc5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:25.938179 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-53d5c7d0-5203-443c-b413-06d0c43e12ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2195dbeb77804451911c7af202cadcc5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:25.940658 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:25.952995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e38cb00-08a5-4bc9-aa88-3be7804119b1 tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:25.956378 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1285/2555] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:04:25 2022] DELETE /identity/v3/users/2195dbeb77804451911c7af202cadcc5 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:25.957938 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f590fe6-0527-45ea-a5bd-0012d470a2e5 tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd992e99ca7745afa886fd06a0f60fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6522666621246c3a37f4b660ce4dbf0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:25.958625 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f590fe6-0527-45ea-a5bd-0012d470a2e5 tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:25.958951 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f590fe6-0527-45ea-a5bd-0012d470a2e5 tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:25.959563 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f590fe6-0527-45ea-a5bd-0012d470a2e5 tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:25.960018 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f590fe6-0527-45ea-a5bd-0012d470a2e5 tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:25.960930 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fd7d1-f9fe-4a88-83cc-f16cc0b26244 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:25.961223 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fd7d1-f9fe-4a88-83cc-f16cc0b26244 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:25.961524 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fd7d1-f9fe-4a88-83cc-f16cc0b26244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:25.961973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f590fe6-0527-45ea-a5bd-0012d470a2e5 tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:25.979660 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-a03fd7d1-f9fe-4a88-83cc-f16cc0b26244 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:25.981558 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a03fd7d1-f9fe-4a88-83cc-f16cc0b26244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:25.995164 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1271/2556] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:04:25 2022] POST /identity/v3/projects => generated 354 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:04:25.996288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f590fe6-0527-45ea-a5bd-0012d470a2e5 tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:25.998592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-8f590fe6-0527-45ea-a5bd-0012d470a2e5 tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:26.066096 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1286/2557] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:04:26.078526 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fd7d1-f9fe-4a88-83cc-f16cc0b26244 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:26.080239 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a03fd7d1-f9fe-4a88-83cc-f16cc0b26244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:26.083809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f590fe6-0527-45ea-a5bd-0012d470a2e5 tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:26.088566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad845bed-5f1d-49b4-a902-694f0164b85b tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd992e99ca7745afa886fd06a0f60fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6522666621246c3a37f4b660ce4dbf0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:26.089224 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad845bed-5f1d-49b4-a902-694f0164b85b tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:26.089536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad845bed-5f1d-49b4-a902-694f0164b85b tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:26.089793 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad845bed-5f1d-49b4-a902-694f0164b85b tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:26.090679 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad845bed-5f1d-49b4-a902-694f0164b85b tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:04:26.090983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad845bed-5f1d-49b4-a902-694f0164b85b tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:26.092774 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad845bed-5f1d-49b4-a902-694f0164b85b tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:26.106208 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1272/2558] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 15:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:04:26.148578 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad845bed-5f1d-49b4-a902-694f0164b85b tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:26.152825 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fd7d1-f9fe-4a88-83cc-f16cc0b26244 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:26.159471 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad845bed-5f1d-49b4-a902-694f0164b85b tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:26.164800 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb244e7b-8544-4485-8e19-39b2efd6887a tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd992e99ca7745afa886fd06a0f60fc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6522666621246c3a37f4b660ce4dbf0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:26.165669 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb244e7b-8544-4485-8e19-39b2efd6887a tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:26.165952 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb244e7b-8544-4485-8e19-39b2efd6887a tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:26.166282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb244e7b-8544-4485-8e19-39b2efd6887a tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] PATH_INFO: `/v3/projects/75b01abcf1284c7d8b3f99f69e0b68a2` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:26.167527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb244e7b-8544-4485-8e19-39b2efd6887a tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] RBAC: Authorizing `identity:delete_project(project_id=75b01abcf1284c7d8b3f99f69e0b68a2)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:26.169507 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb244e7b-8544-4485-8e19-39b2efd6887a tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:26.203212 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:26.210048 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b6836-2205-40dc-a14a-9805b20c6444 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:26.210713 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b6836-2205-40dc-a14a-9805b20c6444 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:26.211019 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b6836-2205-40dc-a14a-9805b20c6444 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:26.211507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b6836-2205-40dc-a14a-9805b20c6444 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:26.223689 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-fb244e7b-8544-4485-8e19-39b2efd6887a tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75b01abcf1284c7d8b3f99f69e0b68a2', 'request_id': 'req-fb244e7b-8544-4485-8e19-39b2efd6887a'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:26.230388 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-fb244e7b-8544-4485-8e19-39b2efd6887a tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75b01abcf1284c7d8b3f99f69e0b68a2', 'request_id': 'req-fb244e7b-8544-4485-8e19-39b2efd6887a'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:26.237783 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-fb244e7b-8544-4485-8e19-39b2efd6887a tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75b01abcf1284c7d8b3f99f69e0b68a2', 'request_id': 'req-fb244e7b-8544-4485-8e19-39b2efd6887a'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:26.252623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-fb244e7b-8544-4485-8e19-39b2efd6887a tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] The token cache is being invalidate because project 75b01abcf1284c7d8b3f99f69e0b68a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:04:26.253131 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-fb244e7b-8544-4485-8e19-39b2efd6887a tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:26.254229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-fb244e7b-8544-4485-8e19-39b2efd6887a tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:26.256265 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1273/2559] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:04:26 2022] DELETE /identity/v3/projects/75b01abcf1284c7d8b3f99f69e0b68a2 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:04:26.279087 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b6836-2205-40dc-a14a-9805b20c6444 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:26.280633 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b6836-2205-40dc-a14a-9805b20c6444 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:26.285809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:26.293053 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1287/2560] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:04:26.487871 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb244e7b-8544-4485-8e19-39b2efd6887a tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:26.490082 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-fb244e7b-8544-4485-8e19-39b2efd6887a tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:26.567470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb244e7b-8544-4485-8e19-39b2efd6887a tempest-VolumeQuotaClassesTest-192050595 tempest-VolumeQuotaClassesTest-192050595-project-admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:26.572444 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da7cd1-0bf3-46ef-858c-968b6cb1fcb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:26.573877 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da7cd1-0bf3-46ef-858c-968b6cb1fcb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:26.574200 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da7cd1-0bf3-46ef-858c-968b6cb1fcb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:26.574767 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da7cd1-0bf3-46ef-858c-968b6cb1fcb6 admin admin] PATH_INFO: `/v3/users/008eb91ab3fe4c4e9e6d8771d42c2bc3` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:26.576390 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da7cd1-0bf3-46ef-858c-968b6cb1fcb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=008eb91ab3fe4c4e9e6d8771d42c2bc3)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:26.577940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da7cd1-0bf3-46ef-858c-968b6cb1fcb6 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:26.599384 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b6836-2205-40dc-a14a-9805b20c6444 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:26.601199 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7a8b6836-2205-40dc-a14a-9805b20c6444 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:26.619906 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b6836-2205-40dc-a14a-9805b20c6444 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:26.626390 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68f91f-c507-49ce-b07f-32629a486921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:26.627472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68f91f-c507-49ce-b07f-32629a486921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:26.627828 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68f91f-c507-49ce-b07f-32629a486921 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:26.628174 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68f91f-c507-49ce-b07f-32629a486921 admin admin] PATH_INFO: `/v3/projects/750ba9582a504688b8beb0dae10297e9` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:26.630917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68f91f-c507-49ce-b07f-32629a486921 admin admin] RBAC: Authorizing `identity:delete_project(project_id=750ba9582a504688b8beb0dae10297e9)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:26.634971 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68f91f-c507-49ce-b07f-32629a486921 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:26.641808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-95da7cd1-0bf3-46ef-858c-968b6cb1fcb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '008eb91ab3fe4c4e9e6d8771d42c2bc3'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:26.655327 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-95da7cd1-0bf3-46ef-858c-968b6cb1fcb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '008eb91ab3fe4c4e9e6d8771d42c2bc3'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:26.663425 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-95da7cd1-0bf3-46ef-858c-968b6cb1fcb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '008eb91ab3fe4c4e9e6d8771d42c2bc3'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:26.664187 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-95da7cd1-0bf3-46ef-858c-968b6cb1fcb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '008eb91ab3fe4c4e9e6d8771d42c2bc3'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:26.664804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-95da7cd1-0bf3-46ef-858c-968b6cb1fcb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '008eb91ab3fe4c4e9e6d8771d42c2bc3'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:26.679910 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1274/2561] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:04:26 2022] DELETE /identity/v3/users/008eb91ab3fe4c4e9e6d8771d42c2bc3 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:26.684743 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd8b18-6bf5-4616-b0bb-3c3b17b627ef None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:26.685197 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd8b18-6bf5-4616-b0bb-3c3b17b627ef None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:26.685318 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd8b18-6bf5-4616-b0bb-3c3b17b627ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:26.694879 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fc68f91f-c507-49ce-b07f-32629a486921 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '750ba9582a504688b8beb0dae10297e9', 'request_id': 'req-fc68f91f-c507-49ce-b07f-32629a486921'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:26.702272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fc68f91f-c507-49ce-b07f-32629a486921 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '750ba9582a504688b8beb0dae10297e9', 'request_id': 'req-fc68f91f-c507-49ce-b07f-32629a486921'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:26.704933 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-26fd8b18-6bf5-4616-b0bb-3c3b17b627ef None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:26.706210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-26fd8b18-6bf5-4616-b0bb-3c3b17b627ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:26.710990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fc68f91f-c507-49ce-b07f-32629a486921 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '750ba9582a504688b8beb0dae10297e9', 'request_id': 'req-fc68f91f-c507-49ce-b07f-32629a486921'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:26.726234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fc68f91f-c507-49ce-b07f-32629a486921 admin admin] The token cache is being invalidate because project 750ba9582a504688b8beb0dae10297e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:04:26.726533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fc68f91f-c507-49ce-b07f-32629a486921 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:26.727604 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fc68f91f-c507-49ce-b07f-32629a486921 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:26.730935 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1288/2562] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:04:26 2022] DELETE /identity/v3/projects/750ba9582a504688b8beb0dae10297e9 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:26.738174 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d60dfb-1c4a-4e17-bfc3-90f9b9cf607d None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:26.739309 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d60dfb-1c4a-4e17-bfc3-90f9b9cf607d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:26.739309 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d60dfb-1c4a-4e17-bfc3-90f9b9cf607d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:26.741728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:26.763082 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-60d60dfb-1c4a-4e17-bfc3-90f9b9cf607d None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:26.764292 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-60d60dfb-1c4a-4e17-bfc3-90f9b9cf607d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:26.776211 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1275/2563] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:04:26.786512 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd8b18-6bf5-4616-b0bb-3c3b17b627ef None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:26.788383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-26fd8b18-6bf5-4616-b0bb-3c3b17b627ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:26.828855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1289/2564] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:26.836533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3206a5-56a4-45f7-8d13-2a639d24aeff None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:26.836823 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3206a5-56a4-45f7-8d13-2a639d24aeff None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:26.837106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3206a5-56a4-45f7-8d13-2a639d24aeff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:26.856321 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-0e3206a5-56a4-45f7-8d13-2a639d24aeff None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:26.857530 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-0e3206a5-56a4-45f7-8d13-2a639d24aeff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:26.859020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd8b18-6bf5-4616-b0bb-3c3b17b627ef None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:26.864136 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689bcfc-9b4c-4f2f-a8a8-4bc41cb8d64b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:26.864856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689bcfc-9b4c-4f2f-a8a8-4bc41cb8d64b service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:26.865272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689bcfc-9b4c-4f2f-a8a8-4bc41cb8d64b service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:26.865620 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689bcfc-9b4c-4f2f-a8a8-4bc41cb8d64b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:26.867788 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1290/2565] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:04:26.871918 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3206a5-56a4-45f7-8d13-2a639d24aeff None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:26.878035 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689bcfc-9b4c-4f2f-a8a8-4bc41cb8d64b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:26.880034 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689bcfc-9b4c-4f2f-a8a8-4bc41cb8d64b service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:26.882835 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3206a5-56a4-45f7-8d13-2a639d24aeff None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:26.888540 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334a1df-14a2-4d09-8965-bc41180f9e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:26.889241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334a1df-14a2-4d09-8965-bc41180f9e1d admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:26.889521 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334a1df-14a2-4d09-8965-bc41180f9e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:26.889806 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334a1df-14a2-4d09-8965-bc41180f9e1d admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:26.890566 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334a1df-14a2-4d09-8965-bc41180f9e1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:04:26.890874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334a1df-14a2-4d09-8965-bc41180f9e1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:26.892540 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334a1df-14a2-4d09-8965-bc41180f9e1d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:26.893408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1276/2566] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:04:26.912270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1291/2567] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 15:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:04:26.913423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689bcfc-9b4c-4f2f-a8a8-4bc41cb8d64b service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:26.924118 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689bcfc-9b4c-4f2f-a8a8-4bc41cb8d64b service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:26.928753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa6d9c-7fbe-48b9-abef-d8409560200d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:26.929403 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa6d9c-7fbe-48b9-abef-d8409560200d admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:26.929689 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa6d9c-7fbe-48b9-abef-d8409560200d admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:26.929957 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa6d9c-7fbe-48b9-abef-d8409560200d admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:26.931136 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fa6d9c-7fbe-48b9-abef-d8409560200d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:26.932582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fa6d9c-7fbe-48b9-abef-d8409560200d admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:26.963308 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1277/2568] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:04:26 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:26.964827 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334a1df-14a2-4d09-8965-bc41180f9e1d admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:26.976183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334a1df-14a2-4d09-8965-bc41180f9e1d admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:27.041648 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bad664-c598-43ae-8b03-33c8e00bd20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:27.042507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bad664-c598-43ae-8b03-33c8e00bd20c admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:27.042846 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bad664-c598-43ae-8b03-33c8e00bd20c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:27.043109 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bad664-c598-43ae-8b03-33c8e00bd20c admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:27.044313 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bad664-c598-43ae-8b03-33c8e00bd20c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:27.046563 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bad664-c598-43ae-8b03-33c8e00bd20c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:27.063937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:27.080724 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1292/2569] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:04:26 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:27.085175 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa6d9c-7fbe-48b9-abef-d8409560200d admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:27.096140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa6d9c-7fbe-48b9-abef-d8409560200d admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:27.100812 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007c1a5-fbbd-475c-8446-4a87b3ac577c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:27.101479 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007c1a5-fbbd-475c-8446-4a87b3ac577c admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:27.101758 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007c1a5-fbbd-475c-8446-4a87b3ac577c admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:27.102051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007c1a5-fbbd-475c-8446-4a87b3ac577c admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:27.103160 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b007c1a5-fbbd-475c-8446-4a87b3ac577c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:27.104771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b007c1a5-fbbd-475c-8446-4a87b3ac577c admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:27.120886 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1278/2570] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:27.124953 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bad664-c598-43ae-8b03-33c8e00bd20c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:27.136626 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bad664-c598-43ae-8b03-33c8e00bd20c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:27.141535 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c06e0f-e624-4512-bf6b-2bcde804cd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:27.142250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c06e0f-e624-4512-bf6b-2bcde804cd7d admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:27.142625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c06e0f-e624-4512-bf6b-2bcde804cd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:27.142839 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c06e0f-e624-4512-bf6b-2bcde804cd7d admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:27.143718 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c06e0f-e624-4512-bf6b-2bcde804cd7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:27.145525 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c06e0f-e624-4512-bf6b-2bcde804cd7d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:27.157596 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1293/2571] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:04:27.161824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007c1a5-fbbd-475c-8446-4a87b3ac577c admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:27.172380 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007c1a5-fbbd-475c-8446-4a87b3ac577c admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:27.177750 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b910bb-bdd8-424c-9b33-7da7bdae5216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:27.178701 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b910bb-bdd8-424c-9b33-7da7bdae5216 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:27.178976 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b910bb-bdd8-424c-9b33-7da7bdae5216 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:27.179302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b910bb-bdd8-424c-9b33-7da7bdae5216 admin admin] PATH_INFO: `/v3/projects/3e75f194e3364498ba51c12bb3b77e6e/users/885084f0b23745ffa0ff00ef9a0b70c2/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:27.194817 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b910bb-bdd8-424c-9b33-7da7bdae5216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e75f194e3364498ba51c12bb3b77e6e, user_id=885084f0b23745ffa0ff00ef9a0b70c2, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:27.197930 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b910bb-bdd8-424c-9b33-7da7bdae5216 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:27.212072 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1279/2572] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:27 2022] PUT /identity/v3/projects/3e75f194e3364498ba51c12bb3b77e6e/users/885084f0b23745ffa0ff00ef9a0b70c2/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:27.223684 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01b610-62bc-468b-a084-ffdddf02bc65 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:27.224003 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01b610-62bc-468b-a084-ffdddf02bc65 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:27.224248 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01b610-62bc-468b-a084-ffdddf02bc65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:27.252611 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b910bb-bdd8-424c-9b33-7da7bdae5216 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:27.252827 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-2a01b610-62bc-468b-a084-ffdddf02bc65 None None] MFA Rules not processed for user `885084f0b23745ffa0ff00ef9a0b70c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:27.253927 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2a01b610-62bc-468b-a084-ffdddf02bc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:27.254633 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-79b910bb-bdd8-424c-9b33-7da7bdae5216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:27.324763 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1294/2573] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:27.325747 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b910bb-bdd8-424c-9b33-7da7bdae5216 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:27.331280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68298793-5c60-43a1-b447-61384e158dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:27.332209 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68298793-5c60-43a1-b447-61384e158dd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:27.332500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68298793-5c60-43a1-b447-61384e158dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:27.332783 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68298793-5c60-43a1-b447-61384e158dd0 admin admin] PATH_INFO: `/v3/projects/a71ae27fc182483884e94bc7435a99df` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:27.333983 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68298793-5c60-43a1-b447-61384e158dd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a71ae27fc182483884e94bc7435a99df)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:27.335965 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68298793-5c60-43a1-b447-61384e158dd0 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:27.336325 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01b610-62bc-468b-a084-ffdddf02bc65 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:27.337801 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2a01b610-62bc-468b-a084-ffdddf02bc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:27.360303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:27.385499 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-68298793-5c60-43a1-b447-61384e158dd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a71ae27fc182483884e94bc7435a99df', 'request_id': 'req-68298793-5c60-43a1-b447-61384e158dd0'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:27.391666 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-68298793-5c60-43a1-b447-61384e158dd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a71ae27fc182483884e94bc7435a99df', 'request_id': 'req-68298793-5c60-43a1-b447-61384e158dd0'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:27.398862 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01b610-62bc-468b-a084-ffdddf02bc65 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:27.399906 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-68298793-5c60-43a1-b447-61384e158dd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a71ae27fc182483884e94bc7435a99df', 'request_id': 'req-68298793-5c60-43a1-b447-61384e158dd0'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:27.412734 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-68298793-5c60-43a1-b447-61384e158dd0 admin admin] The token cache is being invalidate because project a71ae27fc182483884e94bc7435a99df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:04:27.413017 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-68298793-5c60-43a1-b447-61384e158dd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:27.413655 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-68298793-5c60-43a1-b447-61384e158dd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:27.415246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1280/2574] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:04:27 2022] DELETE /identity/v3/projects/a71ae27fc182483884e94bc7435a99df => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:04:27.418981 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68298793-5c60-43a1-b447-61384e158dd0 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:27.420379 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-68298793-5c60-43a1-b447-61384e158dd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:27.457433 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9163b82-f6a8-45f2-a270-302bccc458a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:27.458165 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9163b82-f6a8-45f2-a270-302bccc458a5 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:27.458491 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9163b82-f6a8-45f2-a270-302bccc458a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:27.458803 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9163b82-f6a8-45f2-a270-302bccc458a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:27.460825 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d9163b82-f6a8-45f2-a270-302bccc458a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:27.485985 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68298793-5c60-43a1-b447-61384e158dd0 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:27.491052 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09713ce9-aade-4aa5-aeba-747c49d5e636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:27.491952 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09713ce9-aade-4aa5-aeba-747c49d5e636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:27.492232 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09713ce9-aade-4aa5-aeba-747c49d5e636 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:27.492504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09713ce9-aade-4aa5-aeba-747c49d5e636 admin admin] PATH_INFO: `/v3/users/fd992e99ca7745afa886fd06a0f60fc3` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:27.493715 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09713ce9-aade-4aa5-aeba-747c49d5e636 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd992e99ca7745afa886fd06a0f60fc3)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:27.495685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09713ce9-aade-4aa5-aeba-747c49d5e636 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:27.504788 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:27.537116 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9163b82-f6a8-45f2-a270-302bccc458a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:27.539597 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9163b82-f6a8-45f2-a270-302bccc458a5 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:27.551653 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1295/2575] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:04:27.555883 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-09713ce9-aade-4aa5-aeba-747c49d5e636 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd992e99ca7745afa886fd06a0f60fc3'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:27.569647 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-09713ce9-aade-4aa5-aeba-747c49d5e636 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd992e99ca7745afa886fd06a0f60fc3'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:27.577714 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-09713ce9-aade-4aa5-aeba-747c49d5e636 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd992e99ca7745afa886fd06a0f60fc3'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:27.578419 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-09713ce9-aade-4aa5-aeba-747c49d5e636 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd992e99ca7745afa886fd06a0f60fc3'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:27.579044 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-09713ce9-aade-4aa5-aeba-747c49d5e636 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd992e99ca7745afa886fd06a0f60fc3'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:27.592460 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1281/2576] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:04:27 2022] DELETE /identity/v3/users/fd992e99ca7745afa886fd06a0f60fc3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:04:27.914324 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9163b82-f6a8-45f2-a270-302bccc458a5 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:27.929474 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9163b82-f6a8-45f2-a270-302bccc458a5 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:27.998332 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7708b3-f207-4ca1-a6d1-8e7225b63f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:27.999793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7708b3-f207-4ca1-a6d1-8e7225b63f8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:28.000076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7708b3-f207-4ca1-a6d1-8e7225b63f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:28.000352 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7708b3-f207-4ca1-a6d1-8e7225b63f8d admin admin] PATH_INFO: `/v3/projects/b6522666621246c3a37f4b660ce4dbf0` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:28.001537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7708b3-f207-4ca1-a6d1-8e7225b63f8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6522666621246c3a37f4b660ce4dbf0)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:28.003487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7708b3-f207-4ca1-a6d1-8e7225b63f8d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:28.050791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8d7708b3-f207-4ca1-a6d1-8e7225b63f8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6522666621246c3a37f4b660ce4dbf0', 'request_id': 'req-8d7708b3-f207-4ca1-a6d1-8e7225b63f8d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:28.056576 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8d7708b3-f207-4ca1-a6d1-8e7225b63f8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6522666621246c3a37f4b660ce4dbf0', 'request_id': 'req-8d7708b3-f207-4ca1-a6d1-8e7225b63f8d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:28.063757 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8d7708b3-f207-4ca1-a6d1-8e7225b63f8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6522666621246c3a37f4b660ce4dbf0', 'request_id': 'req-8d7708b3-f207-4ca1-a6d1-8e7225b63f8d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:28.077557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8d7708b3-f207-4ca1-a6d1-8e7225b63f8d admin admin] The token cache is being invalidate because project b6522666621246c3a37f4b660ce4dbf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:04:28.077854 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8d7708b3-f207-4ca1-a6d1-8e7225b63f8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:28.078562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8d7708b3-f207-4ca1-a6d1-8e7225b63f8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:28.080367 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1296/2577] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:04:27 2022] DELETE /identity/v3/projects/b6522666621246c3a37f4b660ce4dbf0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:28.085458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814f250-182e-449d-891e-fc2f2b0021cb None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:28.085927 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814f250-182e-449d-891e-fc2f2b0021cb None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:28.086179 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814f250-182e-449d-891e-fc2f2b0021cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:28.104631 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-9814f250-182e-449d-891e-fc2f2b0021cb None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:28.105837 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-9814f250-182e-449d-891e-fc2f2b0021cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:28.132381 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:28.172424 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1282/2578] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:28.180055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc3b2e-ee9b-4369-b3ee-5193606e225c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:28.180551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc3b2e-ee9b-4369-b3ee-5193606e225c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:28.180646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc3b2e-ee9b-4369-b3ee-5193606e225c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:28.199593 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-98bc3b2e-ee9b-4369-b3ee-5193606e225c None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:28.200803 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-98bc3b2e-ee9b-4369-b3ee-5193606e225c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:28.209732 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1297/2579] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:28.214055 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814f250-182e-449d-891e-fc2f2b0021cb None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:28.223780 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814f250-182e-449d-891e-fc2f2b0021cb None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:28.229170 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8bd50-aee0-4d8a-974b-2bf2960d7b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:28.229863 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8bd50-aee0-4d8a-974b-2bf2960d7b01 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:28.230177 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8bd50-aee0-4d8a-974b-2bf2960d7b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:28.230459 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8bd50-aee0-4d8a-974b-2bf2960d7b01 admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:28.231220 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8bd50-aee0-4d8a-974b-2bf2960d7b01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:04:28.231527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8bd50-aee0-4d8a-974b-2bf2960d7b01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:28.233704 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8bd50-aee0-4d8a-974b-2bf2960d7b01 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:28.245748 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1283/2580] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 15:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:04:28.250648 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc3b2e-ee9b-4369-b3ee-5193606e225c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:28.260473 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc3b2e-ee9b-4369-b3ee-5193606e225c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:28.265199 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1adbaf-59e8-4ebe-ad2f-847f3c6f081a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:28.265855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1adbaf-59e8-4ebe-ad2f-847f3c6f081a admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:28.266188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1adbaf-59e8-4ebe-ad2f-847f3c6f081a admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:28.266476 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1adbaf-59e8-4ebe-ad2f-847f3c6f081a admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:28.267204 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1adbaf-59e8-4ebe-ad2f-847f3c6f081a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:28.269148 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1adbaf-59e8-4ebe-ad2f-847f3c6f081a admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:28.296202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1298/2581] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:04:28 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:28.300342 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8bd50-aee0-4d8a-974b-2bf2960d7b01 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:28.310399 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8bd50-aee0-4d8a-974b-2bf2960d7b01 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:28.368825 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f6b64-35cc-4861-9973-aeec13793c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:28.369662 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f6b64-35cc-4861-9973-aeec13793c84 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:28.369947 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f6b64-35cc-4861-9973-aeec13793c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:28.370257 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f6b64-35cc-4861-9973-aeec13793c84 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:28.370973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95f6b64-35cc-4861-9973-aeec13793c84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:28.372882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95f6b64-35cc-4861-9973-aeec13793c84 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:28.393299 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1284/2582] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:04:28 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:28.397516 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1adbaf-59e8-4ebe-ad2f-847f3c6f081a admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:28.409591 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1adbaf-59e8-4ebe-ad2f-847f3c6f081a admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:28.414846 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378832c0-2cf7-4a99-b383-1958dc8baa34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:28.415477 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378832c0-2cf7-4a99-b383-1958dc8baa34 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:28.415763 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378832c0-2cf7-4a99-b383-1958dc8baa34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:28.416038 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378832c0-2cf7-4a99-b383-1958dc8baa34 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:28.416787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378832c0-2cf7-4a99-b383-1958dc8baa34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:28.418494 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378832c0-2cf7-4a99-b383-1958dc8baa34 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:28.430238 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1299/2583] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:28.435020 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f6b64-35cc-4861-9973-aeec13793c84 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:28.445000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f6b64-35cc-4861-9973-aeec13793c84 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:28.449667 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704566f7-a874-42ee-a396-6857c1678792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:28.450354 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704566f7-a874-42ee-a396-6857c1678792 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:28.450634 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704566f7-a874-42ee-a396-6857c1678792 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:28.450909 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704566f7-a874-42ee-a396-6857c1678792 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:28.451630 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704566f7-a874-42ee-a396-6857c1678792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:28.453239 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704566f7-a874-42ee-a396-6857c1678792 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:28.463639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1285/2584] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:28.468303 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378832c0-2cf7-4a99-b383-1958dc8baa34 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:28.478425 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378832c0-2cf7-4a99-b383-1958dc8baa34 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:28.483806 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6413d-dbd9-4992-9be6-6468097ca357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:28.484684 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6413d-dbd9-4992-9be6-6468097ca357 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:28.484967 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6413d-dbd9-4992-9be6-6468097ca357 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:28.485245 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6413d-dbd9-4992-9be6-6468097ca357 admin admin] PATH_INFO: `/v3/projects/841e7ea1a7c84c1f8b4f22cab4db5533/users/c4b32e3886fb487488b715fbf8c22bf2/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:28.499647 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d6413d-dbd9-4992-9be6-6468097ca357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=841e7ea1a7c84c1f8b4f22cab4db5533, user_id=c4b32e3886fb487488b715fbf8c22bf2, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:28.502354 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d6413d-dbd9-4992-9be6-6468097ca357 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:28.515705 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1300/2585] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:28 2022] PUT /identity/v3/projects/841e7ea1a7c84c1f8b4f22cab4db5533/users/c4b32e3886fb487488b715fbf8c22bf2/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:28.525808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28815629-8234-4b33-bb63-a689961b1b2d None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:28.526131 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28815629-8234-4b33-bb63-a689961b1b2d None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:28.526406 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28815629-8234-4b33-bb63-a689961b1b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:28.553402 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-28815629-8234-4b33-bb63-a689961b1b2d None None] MFA Rules not processed for user `c4b32e3886fb487488b715fbf8c22bf2`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:28.554656 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-28815629-8234-4b33-bb63-a689961b1b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:28.617939 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1286/2586] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:28.622608 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6413d-dbd9-4992-9be6-6468097ca357 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:28.632218 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:28.693213 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6413d-dbd9-4992-9be6-6468097ca357 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:28.697922 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d675ca0-fb53-4a89-b945-233c8b2b6e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:28.698612 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d675ca0-fb53-4a89-b945-233c8b2b6e5c admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:28.698896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d675ca0-fb53-4a89-b945-233c8b2b6e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:28.699209 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d675ca0-fb53-4a89-b945-233c8b2b6e5c admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:28.699874 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d675ca0-fb53-4a89-b945-233c8b2b6e5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:28.701832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d675ca0-fb53-4a89-b945-233c8b2b6e5c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:28.727278 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1301/2587] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:04:28 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:28.731486 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28815629-8234-4b33-bb63-a689961b1b2d None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:28.742966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28815629-8234-4b33-bb63-a689961b1b2d None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:28.763604 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:28.801289 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80e185-b15f-4d6c-964b-bd763cb28824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:28.801990 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80e185-b15f-4d6c-964b-bd763cb28824 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:28.802265 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80e185-b15f-4d6c-964b-bd763cb28824 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:28.802559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80e185-b15f-4d6c-964b-bd763cb28824 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:28.803248 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80e185-b15f-4d6c-964b-bd763cb28824 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:28.805167 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80e185-b15f-4d6c-964b-bd763cb28824 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:28.831642 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1287/2588] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:04:28 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:04:28.835668 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d675ca0-fb53-4a89-b945-233c8b2b6e5c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:28.846158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d675ca0-fb53-4a89-b945-233c8b2b6e5c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:28.851057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825dc17-f9ba-4376-a80c-7eaf13e9a0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:28.851744 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825dc17-f9ba-4376-a80c-7eaf13e9a0be admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:28.852068 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825dc17-f9ba-4376-a80c-7eaf13e9a0be admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:28.852404 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825dc17-f9ba-4376-a80c-7eaf13e9a0be admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:28.853310 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825dc17-f9ba-4376-a80c-7eaf13e9a0be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:28.855009 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825dc17-f9ba-4376-a80c-7eaf13e9a0be admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:28.866652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1302/2589] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:04:28.870900 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80e185-b15f-4d6c-964b-bd763cb28824 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:28.881623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80e185-b15f-4d6c-964b-bd763cb28824 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:28.886607 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94485d-4ea0-489b-8917-8f13fc3f744c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:28.887474 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94485d-4ea0-489b-8917-8f13fc3f744c admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:28.887752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94485d-4ea0-489b-8917-8f13fc3f744c admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:28.888023 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94485d-4ea0-489b-8917-8f13fc3f744c admin admin] PATH_INFO: `/v3/projects/08f5528fe3f74299bb11075471215b4d/users/9f0e3812ebc645dfbaec4d4c084934d5/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:28.903249 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94485d-4ea0-489b-8917-8f13fc3f744c admin admin] RBAC: Authorizing `identity:create_grant(project_id=08f5528fe3f74299bb11075471215b4d, user_id=9f0e3812ebc645dfbaec4d4c084934d5, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:28.905946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94485d-4ea0-489b-8917-8f13fc3f744c admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:28.918958 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1288/2590] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:28 2022] PUT /identity/v3/projects/08f5528fe3f74299bb11075471215b4d/users/9f0e3812ebc645dfbaec4d4c084934d5/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:28.923595 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825dc17-f9ba-4376-a80c-7eaf13e9a0be admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:28.933656 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825dc17-f9ba-4376-a80c-7eaf13e9a0be admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:28.997808 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d0c66-0e9d-474d-8465-71ec0da2d5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:28.999166 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d0c66-0e9d-474d-8465-71ec0da2d5f6 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:28.999346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d0c66-0e9d-474d-8465-71ec0da2d5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:28.999622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d0c66-0e9d-474d-8465-71ec0da2d5f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:29.000341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d0c66-0e9d-474d-8465-71ec0da2d5f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:29.001982 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d0c66-0e9d-474d-8465-71ec0da2d5f6 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:29.012271 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1303/2591] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:29.016525 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94485d-4ea0-489b-8917-8f13fc3f744c admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:29.027879 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94485d-4ea0-489b-8917-8f13fc3f744c admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:29.033661 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb62e6e-71a7-4a12-b16b-6073b48ca959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:29.034570 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb62e6e-71a7-4a12-b16b-6073b48ca959 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:29.034852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb62e6e-71a7-4a12-b16b-6073b48ca959 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:29.035127 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb62e6e-71a7-4a12-b16b-6073b48ca959 admin admin] PATH_INFO: `/v3/projects/08f5528fe3f74299bb11075471215b4d/users/9f0e3812ebc645dfbaec4d4c084934d5/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:29.037277 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb62e6e-71a7-4a12-b16b-6073b48ca959 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08f5528fe3f74299bb11075471215b4d, user_id=9f0e3812ebc645dfbaec4d4c084934d5, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:29.039994 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb62e6e-71a7-4a12-b16b-6073b48ca959 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:29.056771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1289/2592] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:29 2022] PUT /identity/v3/projects/08f5528fe3f74299bb11075471215b4d/users/9f0e3812ebc645dfbaec4d4c084934d5/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:29.068589 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1ba21-fe54-4a83-9add-037d2f82935d None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:29.068968 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1ba21-fe54-4a83-9add-037d2f82935d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:29.069254 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1ba21-fe54-4a83-9add-037d2f82935d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:29.098337 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-e1f1ba21-fe54-4a83-9add-037d2f82935d None None] MFA Rules not processed for user `9f0e3812ebc645dfbaec4d4c084934d5`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:29.099593 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e1f1ba21-fe54-4a83-9add-037d2f82935d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:29.186632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1304/2593] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:29.199014 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb62e6e-71a7-4a12-b16b-6073b48ca959 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:29.200816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-1bb62e6e-71a7-4a12-b16b-6073b48ca959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:29.283475 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb62e6e-71a7-4a12-b16b-6073b48ca959 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:29.288349 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057646d9-6870-4bf1-8bd2-b86ff479cd8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:29.289345 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057646d9-6870-4bf1-8bd2-b86ff479cd8b service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:29.289861 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057646d9-6870-4bf1-8bd2-b86ff479cd8b service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:29.290265 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057646d9-6870-4bf1-8bd2-b86ff479cd8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:29.301845 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057646d9-6870-4bf1-8bd2-b86ff479cd8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:29.303380 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057646d9-6870-4bf1-8bd2-b86ff479cd8b service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:29.314507 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1290/2594] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:29.471485 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1ba21-fe54-4a83-9add-037d2f82935d None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:29.475821 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:29.540484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1ba21-fe54-4a83-9add-037d2f82935d None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:29.545067 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f30235-fc2e-49d5-8832-541b0b05eef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:29.545970 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f30235-fc2e-49d5-8832-541b0b05eef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:29.546286 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f30235-fc2e-49d5-8832-541b0b05eef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:29.546561 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f30235-fc2e-49d5-8832-541b0b05eef9 admin admin] PATH_INFO: `/v3/users/c4b32e3886fb487488b715fbf8c22bf2` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:29.547749 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f30235-fc2e-49d5-8832-541b0b05eef9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4b32e3886fb487488b715fbf8c22bf2)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:29.549694 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f30235-fc2e-49d5-8832-541b0b05eef9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:29.595351 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-77f30235-fc2e-49d5-8832-541b0b05eef9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4b32e3886fb487488b715fbf8c22bf2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:29.607397 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-77f30235-fc2e-49d5-8832-541b0b05eef9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4b32e3886fb487488b715fbf8c22bf2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:29.614353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-77f30235-fc2e-49d5-8832-541b0b05eef9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4b32e3886fb487488b715fbf8c22bf2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:29.614964 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-77f30235-fc2e-49d5-8832-541b0b05eef9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4b32e3886fb487488b715fbf8c22bf2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:29.615565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-77f30235-fc2e-49d5-8832-541b0b05eef9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4b32e3886fb487488b715fbf8c22bf2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:29.627816 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1305/2595] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:04:29 2022] DELETE /identity/v3/users/c4b32e3886fb487488b715fbf8c22bf2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:29.632307 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f01ce-2c0e-4fbf-bd82-6e4ecb547307 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:29.632624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f01ce-2c0e-4fbf-bd82-6e4ecb547307 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:29.632929 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f01ce-2c0e-4fbf-bd82-6e4ecb547307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:29.651443 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-885f01ce-2c0e-4fbf-bd82-6e4ecb547307 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:29.652652 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-885f01ce-2c0e-4fbf-bd82-6e4ecb547307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:29.710734 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1291/2596] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:29.720928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f30235-fc2e-49d5-8832-541b0b05eef9 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:29.722439 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-77f30235-fc2e-49d5-8832-541b0b05eef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:29.781228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f30235-fc2e-49d5-8832-541b0b05eef9 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:29.784887 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0622685-309f-4e4e-ae2f-e9b8cf896422 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:29.785531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0622685-309f-4e4e-ae2f-e9b8cf896422 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:29.785816 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0622685-309f-4e4e-ae2f-e9b8cf896422 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:29.786130 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0622685-309f-4e4e-ae2f-e9b8cf896422 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:29.795624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0622685-309f-4e4e-ae2f-e9b8cf896422 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:29.797113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0622685-309f-4e4e-ae2f-e9b8cf896422 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:29.806791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1306/2597] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:30.080978 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f01ce-2c0e-4fbf-bd82-6e4ecb547307 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:30.082447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-885f01ce-2c0e-4fbf-bd82-6e4ecb547307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:30.094557 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f01ce-2c0e-4fbf-bd82-6e4ecb547307 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:30.099208 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991d0ae-3bc5-4159-bd3e-88fb9a7db273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:30.100063 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991d0ae-3bc5-4159-bd3e-88fb9a7db273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:30.100340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991d0ae-3bc5-4159-bd3e-88fb9a7db273 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:30.100613 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991d0ae-3bc5-4159-bd3e-88fb9a7db273 admin admin] PATH_INFO: `/v3/projects/841e7ea1a7c84c1f8b4f22cab4db5533` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:30.101799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2991d0ae-3bc5-4159-bd3e-88fb9a7db273 admin admin] RBAC: Authorizing `identity:delete_project(project_id=841e7ea1a7c84c1f8b4f22cab4db5533)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:30.103734 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2991d0ae-3bc5-4159-bd3e-88fb9a7db273 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:30.120231 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:30.153482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2991d0ae-3bc5-4159-bd3e-88fb9a7db273 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '841e7ea1a7c84c1f8b4f22cab4db5533', 'request_id': 'req-2991d0ae-3bc5-4159-bd3e-88fb9a7db273'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:30.159306 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2991d0ae-3bc5-4159-bd3e-88fb9a7db273 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '841e7ea1a7c84c1f8b4f22cab4db5533', 'request_id': 'req-2991d0ae-3bc5-4159-bd3e-88fb9a7db273'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:30.165858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2991d0ae-3bc5-4159-bd3e-88fb9a7db273 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '841e7ea1a7c84c1f8b4f22cab4db5533', 'request_id': 'req-2991d0ae-3bc5-4159-bd3e-88fb9a7db273'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:30.178581 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2991d0ae-3bc5-4159-bd3e-88fb9a7db273 admin admin] The token cache is being invalidate because project 841e7ea1a7c84c1f8b4f22cab4db5533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:04:30.178793 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2991d0ae-3bc5-4159-bd3e-88fb9a7db273 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:30.179445 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-2991d0ae-3bc5-4159-bd3e-88fb9a7db273 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:30.181035 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1292/2598] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:04:30 2022] DELETE /identity/v3/projects/841e7ea1a7c84c1f8b4f22cab4db5533 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:30.185141 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0622685-309f-4e4e-ae2f-e9b8cf896422 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:30.195811 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0622685-309f-4e4e-ae2f-e9b8cf896422 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:30.216804 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:30.255757 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219aca5-a1b9-4c65-b303-93da4dcabdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:30.256597 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219aca5-a1b9-4c65-b303-93da4dcabdfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:30.256881 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219aca5-a1b9-4c65-b303-93da4dcabdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:30.257154 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219aca5-a1b9-4c65-b303-93da4dcabdfb admin admin] PATH_INFO: `/v3/users/9f0e3812ebc645dfbaec4d4c084934d5` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:30.258381 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219aca5-a1b9-4c65-b303-93da4dcabdfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f0e3812ebc645dfbaec4d4c084934d5)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:30.260573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219aca5-a1b9-4c65-b303-93da4dcabdfb admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:30.305460 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1219aca5-a1b9-4c65-b303-93da4dcabdfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f0e3812ebc645dfbaec4d4c084934d5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:30.318289 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1219aca5-a1b9-4c65-b303-93da4dcabdfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f0e3812ebc645dfbaec4d4c084934d5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:30.326189 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1219aca5-a1b9-4c65-b303-93da4dcabdfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f0e3812ebc645dfbaec4d4c084934d5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:30.326823 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1219aca5-a1b9-4c65-b303-93da4dcabdfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f0e3812ebc645dfbaec4d4c084934d5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:30.327434 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1219aca5-a1b9-4c65-b303-93da4dcabdfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f0e3812ebc645dfbaec4d4c084934d5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:30.340110 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1307/2599] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:04:30 2022] DELETE /identity/v3/users/9f0e3812ebc645dfbaec4d4c084934d5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:30.750506 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991d0ae-3bc5-4159-bd3e-88fb9a7db273 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:30.752262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-2991d0ae-3bc5-4159-bd3e-88fb9a7db273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:30.771188 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991d0ae-3bc5-4159-bd3e-88fb9a7db273 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:30.837332 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055f4b7-1cac-4442-b909-a4bafd53e9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:30.838264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055f4b7-1cac-4442-b909-a4bafd53e9d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:30.838656 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055f4b7-1cac-4442-b909-a4bafd53e9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:30.838974 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055f4b7-1cac-4442-b909-a4bafd53e9d0 admin admin] PATH_INFO: `/v3/projects/08f5528fe3f74299bb11075471215b4d` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:30.840199 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a055f4b7-1cac-4442-b909-a4bafd53e9d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08f5528fe3f74299bb11075471215b4d)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:30.842816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a055f4b7-1cac-4442-b909-a4bafd53e9d0 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:30.900171 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-a055f4b7-1cac-4442-b909-a4bafd53e9d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08f5528fe3f74299bb11075471215b4d', 'request_id': 'req-a055f4b7-1cac-4442-b909-a4bafd53e9d0'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:30.906491 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-a055f4b7-1cac-4442-b909-a4bafd53e9d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08f5528fe3f74299bb11075471215b4d', 'request_id': 'req-a055f4b7-1cac-4442-b909-a4bafd53e9d0'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:30.913055 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-a055f4b7-1cac-4442-b909-a4bafd53e9d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08f5528fe3f74299bb11075471215b4d', 'request_id': 'req-a055f4b7-1cac-4442-b909-a4bafd53e9d0'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:30.926697 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-a055f4b7-1cac-4442-b909-a4bafd53e9d0 admin admin] The token cache is being invalidate because project 08f5528fe3f74299bb11075471215b4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:04:30.926985 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-a055f4b7-1cac-4442-b909-a4bafd53e9d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:30.927614 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-a055f4b7-1cac-4442-b909-a4bafd53e9d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:30.929172 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1293/2600] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:04:30 2022] DELETE /identity/v3/projects/08f5528fe3f74299bb11075471215b4d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:30.934495 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf8e7d-37ff-4439-b363-165e3d76dc16 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:30.934797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf8e7d-37ff-4439-b363-165e3d76dc16 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:30.935082 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf8e7d-37ff-4439-b363-165e3d76dc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:30.953671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-53bf8e7d-37ff-4439-b363-165e3d76dc16 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:30.954911 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-53bf8e7d-37ff-4439-b363-165e3d76dc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:31.015294 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1308/2601] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:31.022862 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c0b0f-4e28-4124-b01b-c5f103ddad00 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:31.023159 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c0b0f-4e28-4124-b01b-c5f103ddad00 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:31.023440 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c0b0f-4e28-4124-b01b-c5f103ddad00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:31.042648 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-c55c0b0f-4e28-4124-b01b-c5f103ddad00 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:31.043856 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-c55c0b0f-4e28-4124-b01b-c5f103ddad00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:31.051939 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1294/2602] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:31.056378 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf8e7d-37ff-4439-b363-165e3d76dc16 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:31.066168 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf8e7d-37ff-4439-b363-165e3d76dc16 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:31.070785 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37afcfdd-f685-44c0-b25b-16a583710436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:31.071439 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afcfdd-f685-44c0-b25b-16a583710436 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:31.071724 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afcfdd-f685-44c0-b25b-16a583710436 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:31.072000 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afcfdd-f685-44c0-b25b-16a583710436 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:31.072725 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37afcfdd-f685-44c0-b25b-16a583710436 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:04:31.073031 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37afcfdd-f685-44c0-b25b-16a583710436 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:31.074672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37afcfdd-f685-44c0-b25b-16a583710436 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:31.087428 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1309/2603] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 15:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:04:31.091876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c0b0f-4e28-4124-b01b-c5f103ddad00 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:31.105014 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c0b0f-4e28-4124-b01b-c5f103ddad00 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:31.110464 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59548a3d-214e-4e90-a250-a7c8c8e84db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:31.111125 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59548a3d-214e-4e90-a250-a7c8c8e84db5 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:31.111406 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59548a3d-214e-4e90-a250-a7c8c8e84db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:31.111680 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59548a3d-214e-4e90-a250-a7c8c8e84db5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:31.112375 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59548a3d-214e-4e90-a250-a7c8c8e84db5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:31.114400 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59548a3d-214e-4e90-a250-a7c8c8e84db5 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:31.141251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1295/2604] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:04:31 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:31.145958 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37afcfdd-f685-44c0-b25b-16a583710436 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:31.156744 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37afcfdd-f685-44c0-b25b-16a583710436 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:31.186809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:31.215969 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7dac77-5ed5-45ab-82af-a59207462b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:31.216833 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7dac77-5ed5-45ab-82af-a59207462b94 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:31.217194 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7dac77-5ed5-45ab-82af-a59207462b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:31.217548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7dac77-5ed5-45ab-82af-a59207462b94 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:31.218372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7dac77-5ed5-45ab-82af-a59207462b94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:31.220691 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7dac77-5ed5-45ab-82af-a59207462b94 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:31.241296 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1310/2605] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:04:31 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:31.244715 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59548a3d-214e-4e90-a250-a7c8c8e84db5 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:31.252129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:31.256119 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59548a3d-214e-4e90-a250-a7c8c8e84db5 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:31.262712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25fe4bb-ee62-4050-8a87-901cc9b00586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:31.263638 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fe4bb-ee62-4050-8a87-901cc9b00586 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:31.264313 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fe4bb-ee62-4050-8a87-901cc9b00586 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:31.264739 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fe4bb-ee62-4050-8a87-901cc9b00586 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:31.265635 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25fe4bb-ee62-4050-8a87-901cc9b00586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:31.268380 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25fe4bb-ee62-4050-8a87-901cc9b00586 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:31.290261 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1296/2606] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:31.294294 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7dac77-5ed5-45ab-82af-a59207462b94 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:31.304999 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7dac77-5ed5-45ab-82af-a59207462b94 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:31.309895 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7e35c-f811-49d8-9cb4-11b3c233301b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:31.310595 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7e35c-f811-49d8-9cb4-11b3c233301b admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:31.310913 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7e35c-f811-49d8-9cb4-11b3c233301b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:31.311195 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7e35c-f811-49d8-9cb4-11b3c233301b admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:31.311958 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7e35c-f811-49d8-9cb4-11b3c233301b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:31.313595 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7e35c-f811-49d8-9cb4-11b3c233301b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:31.324482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1311/2607] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:31.328608 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25fe4bb-ee62-4050-8a87-901cc9b00586 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:31.342824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25fe4bb-ee62-4050-8a87-901cc9b00586 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:31.347932 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51ca93-d2ca-4b35-a589-327ff7906e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:31.348820 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51ca93-d2ca-4b35-a589-327ff7906e00 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:31.349101 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51ca93-d2ca-4b35-a589-327ff7906e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:31.349373 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51ca93-d2ca-4b35-a589-327ff7906e00 admin admin] PATH_INFO: `/v3/projects/a85df0cb13b240248cc69e70655a3af0/users/fac23961c9204a98bd0fee883fdd04eb/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:31.365305 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee51ca93-d2ca-4b35-a589-327ff7906e00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a85df0cb13b240248cc69e70655a3af0, user_id=fac23961c9204a98bd0fee883fdd04eb, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:31.368171 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee51ca93-d2ca-4b35-a589-327ff7906e00 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:31.380831 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1297/2608] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:31 2022] PUT /identity/v3/projects/a85df0cb13b240248cc69e70655a3af0/users/fac23961c9204a98bd0fee883fdd04eb/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:04:31.392584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27160c-649a-4fad-b18b-7fd15d6f9230 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:31.392877 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27160c-649a-4fad-b18b-7fd15d6f9230 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:31.393156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27160c-649a-4fad-b18b-7fd15d6f9230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:31.422311 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-ee27160c-649a-4fad-b18b-7fd15d6f9230 None None] MFA Rules not processed for user `fac23961c9204a98bd0fee883fdd04eb`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:31.423531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ee27160c-649a-4fad-b18b-7fd15d6f9230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:31.493770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1312/2609] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:31.498108 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51ca93-d2ca-4b35-a589-327ff7906e00 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:31.560641 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51ca93-d2ca-4b35-a589-327ff7906e00 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:31.565309 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9139a4-ad59-40b1-9073-6fa58b9fc31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:31.565967 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9139a4-ad59-40b1-9073-6fa58b9fc31e admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:31.566284 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9139a4-ad59-40b1-9073-6fa58b9fc31e admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:31.566561 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9139a4-ad59-40b1-9073-6fa58b9fc31e admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:31.567259 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9139a4-ad59-40b1-9073-6fa58b9fc31e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:31.569193 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9139a4-ad59-40b1-9073-6fa58b9fc31e admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:31.594436 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1298/2610] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:04:31 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:31.598598 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27160c-649a-4fad-b18b-7fd15d6f9230 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:31.610904 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27160c-649a-4fad-b18b-7fd15d6f9230 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:31.667079 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05808fca-a9b5-4fb5-8e80-b18b0de24c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:31.667747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05808fca-a9b5-4fb5-8e80-b18b0de24c7b admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:31.668030 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05808fca-a9b5-4fb5-8e80-b18b0de24c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:31.668306 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05808fca-a9b5-4fb5-8e80-b18b0de24c7b admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:31.668994 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05808fca-a9b5-4fb5-8e80-b18b0de24c7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:31.671598 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05808fca-a9b5-4fb5-8e80-b18b0de24c7b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:31.692489 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1313/2611] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:04:31 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:31.696525 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9139a4-ad59-40b1-9073-6fa58b9fc31e admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:31.707436 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9139a4-ad59-40b1-9073-6fa58b9fc31e admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:31.712175 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3826ab91-1950-408d-8864-32fb3b2eac04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:31.712831 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826ab91-1950-408d-8864-32fb3b2eac04 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:31.713115 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826ab91-1950-408d-8864-32fb3b2eac04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:31.713391 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826ab91-1950-408d-8864-32fb3b2eac04 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:31.714149 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3826ab91-1950-408d-8864-32fb3b2eac04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:31.715738 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3826ab91-1950-408d-8864-32fb3b2eac04 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:31.727285 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1299/2612] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:31.731448 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05808fca-a9b5-4fb5-8e80-b18b0de24c7b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:31.741666 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05808fca-a9b5-4fb5-8e80-b18b0de24c7b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:31.746477 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d702a37-fd69-4c12-85f0-752fc5349911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:31.747142 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d702a37-fd69-4c12-85f0-752fc5349911 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:31.747428 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d702a37-fd69-4c12-85f0-752fc5349911 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:31.747708 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d702a37-fd69-4c12-85f0-752fc5349911 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:31.748435 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d702a37-fd69-4c12-85f0-752fc5349911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:31.750073 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d702a37-fd69-4c12-85f0-752fc5349911 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:31.760987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1314/2613] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:31.766354 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3826ab91-1950-408d-8864-32fb3b2eac04 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:31.777437 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3826ab91-1950-408d-8864-32fb3b2eac04 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:31.783201 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcdb3e-82a3-4fd5-bfd2-8c02f8f3b3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:31.784186 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcdb3e-82a3-4fd5-bfd2-8c02f8f3b3c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:31.784534 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcdb3e-82a3-4fd5-bfd2-8c02f8f3b3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:31.784883 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcdb3e-82a3-4fd5-bfd2-8c02f8f3b3c3 admin admin] PATH_INFO: `/v3/projects/4ee42e8dff6a46209a5b4fe3f57c4031/users/57ab0c63c4af4957bc74a44c10dd3ee9/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:31.804287 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbcdb3e-82a3-4fd5-bfd2-8c02f8f3b3c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ee42e8dff6a46209a5b4fe3f57c4031, user_id=57ab0c63c4af4957bc74a44c10dd3ee9, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:31.807979 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbcdb3e-82a3-4fd5-bfd2-8c02f8f3b3c3 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:31.822505 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1300/2614] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:31 2022] PUT /identity/v3/projects/4ee42e8dff6a46209a5b4fe3f57c4031/users/57ab0c63c4af4957bc74a44c10dd3ee9/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:31.839634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6dbe7-b28a-4dea-89f7-b9a587e33e0f None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:31.839997 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6dbe7-b28a-4dea-89f7-b9a587e33e0f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:31.840436 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6dbe7-b28a-4dea-89f7-b9a587e33e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:31.870922 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-31b6dbe7-b28a-4dea-89f7-b9a587e33e0f None None] MFA Rules not processed for user `57ab0c63c4af4957bc74a44c10dd3ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:31.872235 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-31b6dbe7-b28a-4dea-89f7-b9a587e33e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:31.887750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:31.962668 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1315/2615] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:31.967829 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcdb3e-82a3-4fd5-bfd2-8c02f8f3b3c3 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:32.033413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcdb3e-82a3-4fd5-bfd2-8c02f8f3b3c3 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:32.038327 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7dca00-0493-4ab0-8fb4-333f200bebc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:32.038984 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7dca00-0493-4ab0-8fb4-333f200bebc2 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:32.039367 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7dca00-0493-4ab0-8fb4-333f200bebc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:32.039520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7dca00-0493-4ab0-8fb4-333f200bebc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:32.040236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7dca00-0493-4ab0-8fb4-333f200bebc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:32.042084 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7dca00-0493-4ab0-8fb4-333f200bebc2 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:32.068296 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1301/2616] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:04:31 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:04:32.072723 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6dbe7-b28a-4dea-89f7-b9a587e33e0f None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:32.084230 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6dbe7-b28a-4dea-89f7-b9a587e33e0f None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:32.146986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beabee9-f7b6-4c75-8930-f7e3a4ad5bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:32.147785 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beabee9-f7b6-4c75-8930-f7e3a4ad5bc2 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:32.148130 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beabee9-f7b6-4c75-8930-f7e3a4ad5bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:32.148475 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beabee9-f7b6-4c75-8930-f7e3a4ad5bc2 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:32.149220 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beabee9-f7b6-4c75-8930-f7e3a4ad5bc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:32.151233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beabee9-f7b6-4c75-8930-f7e3a4ad5bc2 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:32.173873 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1316/2617] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:04:32 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:32.175551 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7dca00-0493-4ab0-8fb4-333f200bebc2 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:32.182206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:32.188810 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7dca00-0493-4ab0-8fb4-333f200bebc2 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:32.193956 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa6499-a0a5-4b7e-b563-045e8aba5aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:32.194654 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa6499-a0a5-4b7e-b563-045e8aba5aef admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:32.194941 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa6499-a0a5-4b7e-b563-045e8aba5aef admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:32.195219 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa6499-a0a5-4b7e-b563-045e8aba5aef admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:32.195949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa6499-a0a5-4b7e-b563-045e8aba5aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:32.197541 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa6499-a0a5-4b7e-b563-045e8aba5aef admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:32.209115 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1302/2618] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:32.213465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beabee9-f7b6-4c75-8930-f7e3a4ad5bc2 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:32.226377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beabee9-f7b6-4c75-8930-f7e3a4ad5bc2 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:32.231286 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec36bf2-cd01-4740-a982-f2c1f3694d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:32.232178 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec36bf2-cd01-4740-a982-f2c1f3694d38 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:32.232466 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec36bf2-cd01-4740-a982-f2c1f3694d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:32.232776 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec36bf2-cd01-4740-a982-f2c1f3694d38 admin admin] PATH_INFO: `/v3/projects/e71aeadb1fbf4a45bb2e8016f72c9576/users/8fec4c885a394db1832e816256ea0109/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:32.249042 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec36bf2-cd01-4740-a982-f2c1f3694d38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e71aeadb1fbf4a45bb2e8016f72c9576, user_id=8fec4c885a394db1832e816256ea0109, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:32.251780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec36bf2-cd01-4740-a982-f2c1f3694d38 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:32.266668 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1317/2619] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:32 2022] PUT /identity/v3/projects/e71aeadb1fbf4a45bb2e8016f72c9576/users/8fec4c885a394db1832e816256ea0109/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:04:32.271036 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa6499-a0a5-4b7e-b563-045e8aba5aef admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:32.282156 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa6499-a0a5-4b7e-b563-045e8aba5aef admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:32.339593 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41adb077-6e9a-4946-a038-88b4b3d7b8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:32.340174 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adb077-6e9a-4946-a038-88b4b3d7b8f6 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:32.340462 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adb077-6e9a-4946-a038-88b4b3d7b8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:32.340740 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adb077-6e9a-4946-a038-88b4b3d7b8f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:32.341467 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41adb077-6e9a-4946-a038-88b4b3d7b8f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:32.343169 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41adb077-6e9a-4946-a038-88b4b3d7b8f6 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:32.353594 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1303/2620] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:04:32.357757 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec36bf2-cd01-4740-a982-f2c1f3694d38 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:32.368371 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec36bf2-cd01-4740-a982-f2c1f3694d38 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:32.373035 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace960e2-8165-4673-a719-128b14d0b371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:32.374457 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace960e2-8165-4673-a719-128b14d0b371 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:32.374741 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace960e2-8165-4673-a719-128b14d0b371 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:32.375022 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace960e2-8165-4673-a719-128b14d0b371 admin admin] PATH_INFO: `/v3/projects/e71aeadb1fbf4a45bb2e8016f72c9576/users/8fec4c885a394db1832e816256ea0109/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:32.377009 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace960e2-8165-4673-a719-128b14d0b371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e71aeadb1fbf4a45bb2e8016f72c9576, user_id=8fec4c885a394db1832e816256ea0109, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:32.379712 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace960e2-8165-4673-a719-128b14d0b371 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:32.391474 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1318/2621] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:32 2022] PUT /identity/v3/projects/e71aeadb1fbf4a45bb2e8016f72c9576/users/8fec4c885a394db1832e816256ea0109/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:32.401919 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f988cb-29bc-42dc-b7e0-6f2d1de996d1 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:32.402248 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f988cb-29bc-42dc-b7e0-6f2d1de996d1 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:32.402626 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f988cb-29bc-42dc-b7e0-6f2d1de996d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:32.429971 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-63f988cb-29bc-42dc-b7e0-6f2d1de996d1 None None] MFA Rules not processed for user `8fec4c885a394db1832e816256ea0109`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:32.431206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-63f988cb-29bc-42dc-b7e0-6f2d1de996d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:32.510222 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1304/2622] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:32 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:04:32.521837 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace960e2-8165-4673-a719-128b14d0b371 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:32.524179 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ace960e2-8165-4673-a719-128b14d0b371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:32.603067 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace960e2-8165-4673-a719-128b14d0b371 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:32.608407 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364b469-7345-4f75-bd16-35bb4cbfdf45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:32.609054 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364b469-7345-4f75-bd16-35bb4cbfdf45 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:32.609340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364b469-7345-4f75-bd16-35bb4cbfdf45 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:32.609615 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364b469-7345-4f75-bd16-35bb4cbfdf45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:32.621802 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364b469-7345-4f75-bd16-35bb4cbfdf45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:32.623297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364b469-7345-4f75-bd16-35bb4cbfdf45 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:32.634033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1319/2623] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:32 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:04:32.678273 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f988cb-29bc-42dc-b7e0-6f2d1de996d1 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:32.688954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f988cb-29bc-42dc-b7e0-6f2d1de996d1 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:32.692752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43fd79d-8f9b-4d18-95cd-d30f31569228 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:32.693484 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fd79d-8f9b-4d18-95cd-d30f31569228 service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:32.693834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fd79d-8f9b-4d18-95cd-d30f31569228 service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:32.694204 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fd79d-8f9b-4d18-95cd-d30f31569228 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:32.762096 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43fd79d-8f9b-4d18-95cd-d30f31569228 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:32.763551 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43fd79d-8f9b-4d18-95cd-d30f31569228 service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:32.773643 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1305/2624] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:35.761651 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364b469-7345-4f75-bd16-35bb4cbfdf45 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:35.766716 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:35.834498 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364b469-7345-4f75-bd16-35bb4cbfdf45 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:35.849137 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcffcd1-1dfc-434d-b221-efa9db2475e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:35.850121 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcffcd1-1dfc-434d-b221-efa9db2475e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:35.850430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcffcd1-1dfc-434d-b221-efa9db2475e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:35.850715 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcffcd1-1dfc-434d-b221-efa9db2475e5 admin admin] PATH_INFO: `/v3/users/fac23961c9204a98bd0fee883fdd04eb` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:35.852535 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcffcd1-1dfc-434d-b221-efa9db2475e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fac23961c9204a98bd0fee883fdd04eb)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:35.854825 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcffcd1-1dfc-434d-b221-efa9db2475e5 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:35.916515 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-ffcffcd1-1dfc-434d-b221-efa9db2475e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fac23961c9204a98bd0fee883fdd04eb'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:35.931580 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-ffcffcd1-1dfc-434d-b221-efa9db2475e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fac23961c9204a98bd0fee883fdd04eb'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:35.938888 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-ffcffcd1-1dfc-434d-b221-efa9db2475e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fac23961c9204a98bd0fee883fdd04eb'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:35.939541 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-ffcffcd1-1dfc-434d-b221-efa9db2475e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fac23961c9204a98bd0fee883fdd04eb'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:35.940257 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-ffcffcd1-1dfc-434d-b221-efa9db2475e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fac23961c9204a98bd0fee883fdd04eb'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:35.954028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1320/2625] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:04:35 2022] DELETE /identity/v3/users/fac23961c9204a98bd0fee883fdd04eb => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:35.960227 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc57bdb-8b9c-4088-b8af-5e2090a2f7a4 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:35.960227 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc57bdb-8b9c-4088-b8af-5e2090a2f7a4 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:35.960227 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc57bdb-8b9c-4088-b8af-5e2090a2f7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:35.980684 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-fcc57bdb-8b9c-4088-b8af-5e2090a2f7a4 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:35.982259 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-fcc57bdb-8b9c-4088-b8af-5e2090a2f7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:35.994344 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:35.998274 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0121c-44c9-4311-b98a-ca5b2ed3b826 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:35.998597 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0121c-44c9-4311-b98a-ca5b2ed3b826 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:35.998879 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0121c-44c9-4311-b98a-ca5b2ed3b826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:36.018587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-88d0121c-44c9-4311-b98a-ca5b2ed3b826 None None] MFA Rules not processed for user `b2935579c8554bed8b1e211c47548ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:36.019943 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-88d0121c-44c9-4311-b98a-ca5b2ed3b826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:36.052133 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1306/2626] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:36.062787 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc57bdb-8b9c-4088-b8af-5e2090a2f7a4 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:36.064207 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-fcc57bdb-8b9c-4088-b8af-5e2090a2f7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:36.080085 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1321/2627] 10.222.0.78 () {58 vars in 1009 bytes} [Wed May 25 15:04:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:36.131743 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0121c-44c9-4311-b98a-ca5b2ed3b826 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:36.133889 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-88d0121c-44c9-4311-b98a-ca5b2ed3b826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:36.139767 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc57bdb-8b9c-4088-b8af-5e2090a2f7a4 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:36.144434 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab378b-4685-4666-ae46-8a2673212dd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:36.145610 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab378b-4685-4666-ae46-8a2673212dd6 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:36.146100 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab378b-4685-4666-ae46-8a2673212dd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:36.146387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab378b-4685-4666-ae46-8a2673212dd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:36.157133 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab378b-4685-4666-ae46-8a2673212dd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:36.158812 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab378b-4685-4666-ae46-8a2673212dd6 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:36.167932 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:36.169030 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1307/2628] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:04:36.209777 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0121c-44c9-4311-b98a-ca5b2ed3b826 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:36.216121 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55443df-76bf-4f0c-84c8-c6cd7f630789 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe4a2d106d84fc9ac65c046ba675353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:36.216726 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55443df-76bf-4f0c-84c8-c6cd7f630789 service swift] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:36.217072 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55443df-76bf-4f0c-84c8-c6cd7f630789 service swift] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:36.217910 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55443df-76bf-4f0c-84c8-c6cd7f630789 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:36.227464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55443df-76bf-4f0c-84c8-c6cd7f630789 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:36.229121 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55443df-76bf-4f0c-84c8-c6cd7f630789 service swift] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:36.238539 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1322/2629] 10.222.0.78 () {60 vars in 1384 bytes} [Wed May 25 15:04:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 15:04:36.495863 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab378b-4685-4666-ae46-8a2673212dd6 service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:36.497465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-6dab378b-4685-4666-ae46-8a2673212dd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:36.523276 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab378b-4685-4666-ae46-8a2673212dd6 service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:36.535703 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179757f-bea7-4ab5-b0d5-4ad90f3f2547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:36.536894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179757f-bea7-4ab5-b0d5-4ad90f3f2547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:36.537327 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179757f-bea7-4ab5-b0d5-4ad90f3f2547 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:36.537698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179757f-bea7-4ab5-b0d5-4ad90f3f2547 admin admin] PATH_INFO: `/v3/projects/a85df0cb13b240248cc69e70655a3af0` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:36.539384 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179757f-bea7-4ab5-b0d5-4ad90f3f2547 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a85df0cb13b240248cc69e70655a3af0)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:36.542118 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179757f-bea7-4ab5-b0d5-4ad90f3f2547 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:36.621688 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-9179757f-bea7-4ab5-b0d5-4ad90f3f2547 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a85df0cb13b240248cc69e70655a3af0', 'request_id': 'req-9179757f-bea7-4ab5-b0d5-4ad90f3f2547'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:36.640120 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-9179757f-bea7-4ab5-b0d5-4ad90f3f2547 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a85df0cb13b240248cc69e70655a3af0', 'request_id': 'req-9179757f-bea7-4ab5-b0d5-4ad90f3f2547'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:36.653472 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-9179757f-bea7-4ab5-b0d5-4ad90f3f2547 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a85df0cb13b240248cc69e70655a3af0', 'request_id': 'req-9179757f-bea7-4ab5-b0d5-4ad90f3f2547'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:36.679913 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-9179757f-bea7-4ab5-b0d5-4ad90f3f2547 admin admin] The token cache is being invalidate because project a85df0cb13b240248cc69e70655a3af0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:04:36.681414 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-9179757f-bea7-4ab5-b0d5-4ad90f3f2547 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:36.682200 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-9179757f-bea7-4ab5-b0d5-4ad90f3f2547 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:36.688011 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1308/2630] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:04:36 2022] DELETE /identity/v3/projects/a85df0cb13b240248cc69e70655a3af0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:36.688316 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55443df-76bf-4f0c-84c8-c6cd7f630789 service swift] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:36.702600 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55443df-76bf-4f0c-84c8-c6cd7f630789 service swift] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:36.807593 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e9ab9-c7c9-4599-a9d7-7dec804f7c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:36.809674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e9ab9-c7c9-4599-a9d7-7dec804f7c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:36.810186 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e9ab9-c7c9-4599-a9d7-7dec804f7c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:36.810623 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e9ab9-c7c9-4599-a9d7-7dec804f7c68 admin admin] PATH_INFO: `/v3/users/57ab0c63c4af4957bc74a44c10dd3ee9` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:36.812378 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e9ab9-c7c9-4599-a9d7-7dec804f7c68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57ab0c63c4af4957bc74a44c10dd3ee9)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:36.814757 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e9ab9-c7c9-4599-a9d7-7dec804f7c68 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:36.925728 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-db0e9ab9-c7c9-4599-a9d7-7dec804f7c68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57ab0c63c4af4957bc74a44c10dd3ee9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:36.940393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-db0e9ab9-c7c9-4599-a9d7-7dec804f7c68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57ab0c63c4af4957bc74a44c10dd3ee9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:36.951036 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-db0e9ab9-c7c9-4599-a9d7-7dec804f7c68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57ab0c63c4af4957bc74a44c10dd3ee9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:36.952256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-db0e9ab9-c7c9-4599-a9d7-7dec804f7c68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57ab0c63c4af4957bc74a44c10dd3ee9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:36.952978 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-db0e9ab9-c7c9-4599-a9d7-7dec804f7c68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57ab0c63c4af4957bc74a44c10dd3ee9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:36.970196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1323/2631] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:04:36 2022] DELETE /identity/v3/users/57ab0c63c4af4957bc74a44c10dd3ee9 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:37.334118 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179757f-bea7-4ab5-b0d5-4ad90f3f2547 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:37.336043 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-9179757f-bea7-4ab5-b0d5-4ad90f3f2547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:37.351556 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179757f-bea7-4ab5-b0d5-4ad90f3f2547 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:37.409219 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:37.432777 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba93b25c-71f3-4990-a8ac-27c0003b8ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:37.433852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93b25c-71f3-4990-a8ac-27c0003b8ca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:37.434385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93b25c-71f3-4990-a8ac-27c0003b8ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:37.434691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93b25c-71f3-4990-a8ac-27c0003b8ca4 admin admin] PATH_INFO: `/v3/projects/4ee42e8dff6a46209a5b4fe3f57c4031` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:37.436457 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba93b25c-71f3-4990-a8ac-27c0003b8ca4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ee42e8dff6a46209a5b4fe3f57c4031)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:37.438467 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba93b25c-71f3-4990-a8ac-27c0003b8ca4 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:37.493915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ba93b25c-71f3-4990-a8ac-27c0003b8ca4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ee42e8dff6a46209a5b4fe3f57c4031', 'request_id': 'req-ba93b25c-71f3-4990-a8ac-27c0003b8ca4'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:37.500160 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ba93b25c-71f3-4990-a8ac-27c0003b8ca4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ee42e8dff6a46209a5b4fe3f57c4031', 'request_id': 'req-ba93b25c-71f3-4990-a8ac-27c0003b8ca4'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:37.506898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ba93b25c-71f3-4990-a8ac-27c0003b8ca4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ee42e8dff6a46209a5b4fe3f57c4031', 'request_id': 'req-ba93b25c-71f3-4990-a8ac-27c0003b8ca4'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:37.523595 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ba93b25c-71f3-4990-a8ac-27c0003b8ca4 admin admin] The token cache is being invalidate because project 4ee42e8dff6a46209a5b4fe3f57c4031 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:04:37.523925 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ba93b25c-71f3-4990-a8ac-27c0003b8ca4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:37.524549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ba93b25c-71f3-4990-a8ac-27c0003b8ca4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:37.526164 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1309/2632] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:04:37 2022] DELETE /identity/v3/projects/4ee42e8dff6a46209a5b4fe3f57c4031 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:37.530902 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e9ab9-c7c9-4599-a9d7-7dec804f7c68 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:37.533777 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-db0e9ab9-c7c9-4599-a9d7-7dec804f7c68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:37.549607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e9ab9-c7c9-4599-a9d7-7dec804f7c68 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:37.588726 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:37.628316 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c69b7c-a567-4b31-a216-b24cada76525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:37.629508 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c69b7c-a567-4b31-a216-b24cada76525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:37.629797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c69b7c-a567-4b31-a216-b24cada76525 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:37.630110 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c69b7c-a567-4b31-a216-b24cada76525 admin admin] PATH_INFO: `/v3/users/8fec4c885a394db1832e816256ea0109` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:37.631374 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c69b7c-a567-4b31-a216-b24cada76525 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fec4c885a394db1832e816256ea0109)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:37.633345 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c69b7c-a567-4b31-a216-b24cada76525 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:37.697906 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-08c69b7c-a567-4b31-a216-b24cada76525 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fec4c885a394db1832e816256ea0109'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:37.716395 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-08c69b7c-a567-4b31-a216-b24cada76525 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fec4c885a394db1832e816256ea0109'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:37.729363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-08c69b7c-a567-4b31-a216-b24cada76525 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fec4c885a394db1832e816256ea0109'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:37.730256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-08c69b7c-a567-4b31-a216-b24cada76525 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fec4c885a394db1832e816256ea0109'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:37.731004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-08c69b7c-a567-4b31-a216-b24cada76525 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fec4c885a394db1832e816256ea0109'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:37.755033 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1324/2633] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:04:37 2022] DELETE /identity/v3/users/8fec4c885a394db1832e816256ea0109 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:37.834747 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba93b25c-71f3-4990-a8ac-27c0003b8ca4 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:37.836546 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-ba93b25c-71f3-4990-a8ac-27c0003b8ca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:37.853395 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba93b25c-71f3-4990-a8ac-27c0003b8ca4 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:37.960075 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a8386-a707-4540-81c4-5cd76aee0c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:37.961030 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a8386-a707-4540-81c4-5cd76aee0c6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:37.961313 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a8386-a707-4540-81c4-5cd76aee0c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:37.961678 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a8386-a707-4540-81c4-5cd76aee0c6c admin admin] PATH_INFO: `/v3/users/885084f0b23745ffa0ff00ef9a0b70c2` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:37.974242 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a8386-a707-4540-81c4-5cd76aee0c6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=885084f0b23745ffa0ff00ef9a0b70c2)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:37.977074 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a8386-a707-4540-81c4-5cd76aee0c6c admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:38.059247 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ac9a8386-a707-4540-81c4-5cd76aee0c6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '885084f0b23745ffa0ff00ef9a0b70c2'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:38.075408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ac9a8386-a707-4540-81c4-5cd76aee0c6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '885084f0b23745ffa0ff00ef9a0b70c2'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:38.092150 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ac9a8386-a707-4540-81c4-5cd76aee0c6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '885084f0b23745ffa0ff00ef9a0b70c2'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:38.092852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ac9a8386-a707-4540-81c4-5cd76aee0c6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '885084f0b23745ffa0ff00ef9a0b70c2'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:38.094400 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ac9a8386-a707-4540-81c4-5cd76aee0c6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '885084f0b23745ffa0ff00ef9a0b70c2'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:38.100065 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:38.125803 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1310/2634] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:04:37 2022] DELETE /identity/v3/users/885084f0b23745ffa0ff00ef9a0b70c2 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:38.127765 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67bf50-8592-4bb8-a747-bc670cd97620 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:38.128407 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67bf50-8592-4bb8-a747-bc670cd97620 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:38.128519 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67bf50-8592-4bb8-a747-bc670cd97620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:38.147968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a8386-a707-4540-81c4-5cd76aee0c6c admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:38.149831 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-ac9a8386-a707-4540-81c4-5cd76aee0c6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:38.150174 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-db67bf50-8592-4bb8-a747-bc670cd97620 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:38.151371 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-db67bf50-8592-4bb8-a747-bc670cd97620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:38.227146 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1325/2635] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:38.237204 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67bf50-8592-4bb8-a747-bc670cd97620 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:38.239001 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-db67bf50-8592-4bb8-a747-bc670cd97620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:38.282481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a8386-a707-4540-81c4-5cd76aee0c6c admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:38.291937 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca73f1-7da0-4d47-9a95-36c2ee21e76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:38.292881 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca73f1-7da0-4d47-9a95-36c2ee21e76f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:38.294367 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca73f1-7da0-4d47-9a95-36c2ee21e76f admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:38.294658 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca73f1-7da0-4d47-9a95-36c2ee21e76f admin admin] PATH_INFO: `/v3/projects/e71aeadb1fbf4a45bb2e8016f72c9576` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:38.297422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca73f1-7da0-4d47-9a95-36c2ee21e76f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e71aeadb1fbf4a45bb2e8016f72c9576)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:38.299507 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca73f1-7da0-4d47-9a95-36c2ee21e76f admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:38.383617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c3ca73f1-7da0-4d47-9a95-36c2ee21e76f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e71aeadb1fbf4a45bb2e8016f72c9576', 'request_id': 'req-c3ca73f1-7da0-4d47-9a95-36c2ee21e76f'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:38.391523 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c3ca73f1-7da0-4d47-9a95-36c2ee21e76f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e71aeadb1fbf4a45bb2e8016f72c9576', 'request_id': 'req-c3ca73f1-7da0-4d47-9a95-36c2ee21e76f'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:38.399685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c3ca73f1-7da0-4d47-9a95-36c2ee21e76f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e71aeadb1fbf4a45bb2e8016f72c9576', 'request_id': 'req-c3ca73f1-7da0-4d47-9a95-36c2ee21e76f'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:38.409269 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67bf50-8592-4bb8-a747-bc670cd97620 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:38.422605 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c3ca73f1-7da0-4d47-9a95-36c2ee21e76f admin admin] The token cache is being invalidate because project e71aeadb1fbf4a45bb2e8016f72c9576 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:04:38.423519 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c3ca73f1-7da0-4d47-9a95-36c2ee21e76f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:38.424375 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c3ca73f1-7da0-4d47-9a95-36c2ee21e76f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:38.430054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1311/2636] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:04:38 2022] DELETE /identity/v3/projects/e71aeadb1fbf4a45bb2e8016f72c9576 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:38.430672 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:38.432888 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ff4f3-9d0f-4810-86e3-ab6bffc8eb2c None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:38.433273 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ff4f3-9d0f-4810-86e3-ab6bffc8eb2c None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:38.434191 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ff4f3-9d0f-4810-86e3-ab6bffc8eb2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:38.476839 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-e57ff4f3-9d0f-4810-86e3-ab6bffc8eb2c None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:38.478070 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-e57ff4f3-9d0f-4810-86e3-ab6bffc8eb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:38.508785 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c14797-68e3-4b97-a41b-dd1c5ea64c72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:38.510324 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c14797-68e3-4b97-a41b-dd1c5ea64c72 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:38.510609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c14797-68e3-4b97-a41b-dd1c5ea64c72 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:38.510881 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c14797-68e3-4b97-a41b-dd1c5ea64c72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:38.516286 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-51c14797-68e3-4b97-a41b-dd1c5ea64c72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:38.569456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1312/2637] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:38.579241 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd2816-5a6b-4e5f-90d0-e93b87dbe892 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:38.579528 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd2816-5a6b-4e5f-90d0-e93b87dbe892 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:38.579807 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd2816-5a6b-4e5f-90d0-e93b87dbe892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:38.599771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-1dfd2816-5a6b-4e5f-90d0-e93b87dbe892 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:38.601027 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-1dfd2816-5a6b-4e5f-90d0-e93b87dbe892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:38.613821 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1313/2638] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:38.617383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd2816-5a6b-4e5f-90d0-e93b87dbe892 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:38.618497 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c14797-68e3-4b97-a41b-dd1c5ea64c72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:38.620449 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c14797-68e3-4b97-a41b-dd1c5ea64c72 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:38.632435 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd2816-5a6b-4e5f-90d0-e93b87dbe892 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:38.633860 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1326/2639] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:04:38.639046 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a733afc-097c-4e0a-a4e4-ff50f9eb4e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:38.639779 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a733afc-097c-4e0a-a4e4-ff50f9eb4e5e admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:38.640103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a733afc-097c-4e0a-a4e4-ff50f9eb4e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:38.640423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a733afc-097c-4e0a-a4e4-ff50f9eb4e5e admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:38.641163 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a733afc-097c-4e0a-a4e4-ff50f9eb4e5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:04:38.647280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a733afc-097c-4e0a-a4e4-ff50f9eb4e5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:38.650736 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a733afc-097c-4e0a-a4e4-ff50f9eb4e5e admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:38.662687 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:38.667546 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1314/2640] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 15:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:04:38.672811 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c14797-68e3-4b97-a41b-dd1c5ea64c72 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:38.688334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c14797-68e3-4b97-a41b-dd1c5ea64c72 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:38.694920 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a27fb6-58fe-4a2c-97c8-b3b18ebd0fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:38.695744 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a27fb6-58fe-4a2c-97c8-b3b18ebd0fa7 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:38.695986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a27fb6-58fe-4a2c-97c8-b3b18ebd0fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:38.696287 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a27fb6-58fe-4a2c-97c8-b3b18ebd0fa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:38.697115 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a27fb6-58fe-4a2c-97c8-b3b18ebd0fa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:38.699149 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a27fb6-58fe-4a2c-97c8-b3b18ebd0fa7 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:38.742477 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1327/2641] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:04:38 2022] POST /identity/v3/projects => generated 385 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:38.743160 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a733afc-097c-4e0a-a4e4-ff50f9eb4e5e admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:38.760011 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a733afc-097c-4e0a-a4e4-ff50f9eb4e5e admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:38.855067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0378a-9dd1-49bf-bbb0-d3a674169793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:38.855895 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0378a-9dd1-49bf-bbb0-d3a674169793 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:38.856233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0378a-9dd1-49bf-bbb0-d3a674169793 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:38.856667 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0378a-9dd1-49bf-bbb0-d3a674169793 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:38.859198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0378a-9dd1-49bf-bbb0-d3a674169793 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:38.861241 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0378a-9dd1-49bf-bbb0-d3a674169793 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:38.890196 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1315/2642] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:04:38 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:04:38.891690 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a27fb6-58fe-4a2c-97c8-b3b18ebd0fa7 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:38.914800 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a27fb6-58fe-4a2c-97c8-b3b18ebd0fa7 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:38.921740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cd9e4-c6da-4f3a-83de-23b3807e643f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:38.922509 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cd9e4-c6da-4f3a-83de-23b3807e643f admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:38.922812 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cd9e4-c6da-4f3a-83de-23b3807e643f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:38.923294 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cd9e4-c6da-4f3a-83de-23b3807e643f admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:38.924108 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707cd9e4-c6da-4f3a-83de-23b3807e643f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:38.925871 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707cd9e4-c6da-4f3a-83de-23b3807e643f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:38.943262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1328/2643] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:38 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:38.945538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0378a-9dd1-49bf-bbb0-d3a674169793 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:38.956574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0378a-9dd1-49bf-bbb0-d3a674169793 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:38.964770 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be1ecc-6ba5-4312-87e6-80d3429c980a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:38.965860 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be1ecc-6ba5-4312-87e6-80d3429c980a admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:38.965860 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be1ecc-6ba5-4312-87e6-80d3429c980a admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:38.966194 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be1ecc-6ba5-4312-87e6-80d3429c980a admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:38.966874 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be1ecc-6ba5-4312-87e6-80d3429c980a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:38.969007 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be1ecc-6ba5-4312-87e6-80d3429c980a admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:38.986048 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1316/2644] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:04:38.987670 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cd9e4-c6da-4f3a-83de-23b3807e643f admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:39.007102 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cd9e4-c6da-4f3a-83de-23b3807e643f admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:39.024150 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0cea3d-ec1c-4b07-93df-6bec230667df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:39.025324 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cea3d-ec1c-4b07-93df-6bec230667df admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:39.027414 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cea3d-ec1c-4b07-93df-6bec230667df admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:39.027987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cea3d-ec1c-4b07-93df-6bec230667df admin admin] PATH_INFO: `/v3/projects/4bac591af3764a5c9747f8236ad98cf7/users/64a6d267d37c4eabaa6053c3dd2fd158/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:39.041110 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be1ecc-6ba5-4312-87e6-80d3429c980a admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:39.043077 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-57be1ecc-6ba5-4312-87e6-80d3429c980a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:39.084210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be1ecc-6ba5-4312-87e6-80d3429c980a admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:39.093586 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0cea3d-ec1c-4b07-93df-6bec230667df admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bac591af3764a5c9747f8236ad98cf7, user_id=64a6d267d37c4eabaa6053c3dd2fd158, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:39.096387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f00dfd-2f79-47c4-9f05-aa416dfe9b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:39.097131 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0cea3d-ec1c-4b07-93df-6bec230667df admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:39.098591 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f00dfd-2f79-47c4-9f05-aa416dfe9b85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:39.098954 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f00dfd-2f79-47c4-9f05-aa416dfe9b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:39.101087 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f00dfd-2f79-47c4-9f05-aa416dfe9b85 admin admin] PATH_INFO: `/v3/projects/3e75f194e3364498ba51c12bb3b77e6e` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:39.104174 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f00dfd-2f79-47c4-9f05-aa416dfe9b85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e75f194e3364498ba51c12bb3b77e6e)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:39.106429 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f00dfd-2f79-47c4-9f05-aa416dfe9b85 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:39.140707 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1329/2645] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:38 2022] PUT /identity/v3/projects/4bac591af3764a5c9747f8236ad98cf7/users/64a6d267d37c4eabaa6053c3dd2fd158/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:39.156624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa9dd9-06c2-4fe9-9777-4e6079f546a0 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:39.157349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa9dd9-06c2-4fe9-9777-4e6079f546a0 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:39.158371 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa9dd9-06c2-4fe9-9777-4e6079f546a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:39.194220 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b4f00dfd-2f79-47c4-9f05-aa416dfe9b85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e75f194e3364498ba51c12bb3b77e6e', 'request_id': 'req-b4f00dfd-2f79-47c4-9f05-aa416dfe9b85'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:39.198736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-c3aa9dd9-06c2-4fe9-9777-4e6079f546a0 None None] MFA Rules not processed for user `64a6d267d37c4eabaa6053c3dd2fd158`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:39.200974 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c3aa9dd9-06c2-4fe9-9777-4e6079f546a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:39.209894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b4f00dfd-2f79-47c4-9f05-aa416dfe9b85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e75f194e3364498ba51c12bb3b77e6e', 'request_id': 'req-b4f00dfd-2f79-47c4-9f05-aa416dfe9b85'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:39.218773 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b4f00dfd-2f79-47c4-9f05-aa416dfe9b85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e75f194e3364498ba51c12bb3b77e6e', 'request_id': 'req-b4f00dfd-2f79-47c4-9f05-aa416dfe9b85'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:39.238475 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b4f00dfd-2f79-47c4-9f05-aa416dfe9b85 admin admin] The token cache is being invalidate because project 3e75f194e3364498ba51c12bb3b77e6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:04:39.238911 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b4f00dfd-2f79-47c4-9f05-aa416dfe9b85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:39.239706 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-b4f00dfd-2f79-47c4-9f05-aa416dfe9b85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:39.241446 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1317/2646] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:04:39 2022] DELETE /identity/v3/projects/3e75f194e3364498ba51c12bb3b77e6e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:04:39.247812 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52918827-a349-4090-85d2-8156c4f5880d None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:39.248220 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52918827-a349-4090-85d2-8156c4f5880d None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:39.248633 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52918827-a349-4090-85d2-8156c4f5880d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:39.272752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-52918827-a349-4090-85d2-8156c4f5880d None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:39.274278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-52918827-a349-4090-85d2-8156c4f5880d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:39.309760 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:39.327596 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1330/2647] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:39.334929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa9dd9-06c2-4fe9-9777-4e6079f546a0 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:39.336670 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c3aa9dd9-06c2-4fe9-9777-4e6079f546a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:39.365503 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1318/2648] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:39.374061 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac14418-f400-433b-b06d-1f9ca68f5a97 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:39.374709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac14418-f400-433b-b06d-1f9ca68f5a97 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:39.375235 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac14418-f400-433b-b06d-1f9ca68f5a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:39.386473 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:39.406740 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-fac14418-f400-433b-b06d-1f9ca68f5a97 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:39.408186 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-fac14418-f400-433b-b06d-1f9ca68f5a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:39.417290 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1319/2649] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:39.432464 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac14418-f400-433b-b06d-1f9ca68f5a97 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:39.444066 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa9dd9-06c2-4fe9-9777-4e6079f546a0 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:39.448382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac14418-f400-433b-b06d-1f9ca68f5a97 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:39.452962 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9675f-649d-4de1-a856-2c480b11ae61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:39.453654 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41ea11-f732-45dc-8547-df8b75fa1186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:39.454331 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9675f-649d-4de1-a856-2c480b11ae61 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:39.454461 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41ea11-f732-45dc-8547-df8b75fa1186 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:39.454610 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41ea11-f732-45dc-8547-df8b75fa1186 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:39.454885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41ea11-f732-45dc-8547-df8b75fa1186 admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:39.455214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9675f-649d-4de1-a856-2c480b11ae61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:39.455614 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41ea11-f732-45dc-8547-df8b75fa1186 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:04:39.455923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41ea11-f732-45dc-8547-df8b75fa1186 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:39.456135 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9675f-649d-4de1-a856-2c480b11ae61 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:39.457017 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9675f-649d-4de1-a856-2c480b11ae61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:39.457518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41ea11-f732-45dc-8547-df8b75fa1186 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:39.459728 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9675f-649d-4de1-a856-2c480b11ae61 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:39.471288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1320/2650] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 15:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:04:39.476587 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41ea11-f732-45dc-8547-df8b75fa1186 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:39.487032 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41ea11-f732-45dc-8547-df8b75fa1186 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:39.492301 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415673b-09c5-4aed-b60e-2ca886865498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:39.493142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415673b-09c5-4aed-b60e-2ca886865498 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:39.493516 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415673b-09c5-4aed-b60e-2ca886865498 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:39.494118 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415673b-09c5-4aed-b60e-2ca886865498 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:39.495168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415673b-09c5-4aed-b60e-2ca886865498 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:39.497500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415673b-09c5-4aed-b60e-2ca886865498 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:39.528850 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1331/2651] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:04:39 2022] POST /identity/v3/projects => generated 383 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:39.531352 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9675f-649d-4de1-a856-2c480b11ae61 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:39.540152 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1321/2652] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:04:39 2022] POST /identity/v3/projects => generated 397 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:39.544061 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415673b-09c5-4aed-b60e-2ca886865498 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:39.625636 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9675f-649d-4de1-a856-2c480b11ae61 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:39.648398 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415673b-09c5-4aed-b60e-2ca886865498 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:39.655580 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7b1f6-057c-4bee-b42e-2718d0497d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:39.656261 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7b1f6-057c-4bee-b42e-2718d0497d1e admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:39.656553 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7b1f6-057c-4bee-b42e-2718d0497d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:39.656834 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7b1f6-057c-4bee-b42e-2718d0497d1e admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:39.657530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7b1f6-057c-4bee-b42e-2718d0497d1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:39.659507 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7b1f6-057c-4bee-b42e-2718d0497d1e admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:39.682351 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1322/2653] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:04:39 2022] POST /identity/v3/users => generated 299 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:04:39.687186 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7b1f6-057c-4bee-b42e-2718d0497d1e admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:39.699156 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7b1f6-057c-4bee-b42e-2718d0497d1e admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:39.705566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0e827-cfc6-46f8-96f1-27bdaca6f284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:39.706991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0e827-cfc6-46f8-96f1-27bdaca6f284 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:39.707430 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0e827-cfc6-46f8-96f1-27bdaca6f284 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:39.707871 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0e827-cfc6-46f8-96f1-27bdaca6f284 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:39.708827 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0e827-cfc6-46f8-96f1-27bdaca6f284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:39.710818 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0e827-cfc6-46f8-96f1-27bdaca6f284 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:39.715990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f1a86-446b-4d58-a8cb-5aaa7c2ef070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:39.716898 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f1a86-446b-4d58-a8cb-5aaa7c2ef070 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:39.717441 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f1a86-446b-4d58-a8cb-5aaa7c2ef070 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:39.717921 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f1a86-446b-4d58-a8cb-5aaa7c2ef070 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:39.718769 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f1a86-446b-4d58-a8cb-5aaa7c2ef070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:39.721081 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f1a86-446b-4d58-a8cb-5aaa7c2ef070 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:39.727528 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1323/2654] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:04:39.731959 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0e827-cfc6-46f8-96f1-27bdaca6f284 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:39.745203 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0e827-cfc6-46f8-96f1-27bdaca6f284 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:39.750162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1332/2655] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:04:39 2022] POST /identity/v3/users => generated 298 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:39.757326 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f1a86-446b-4d58-a8cb-5aaa7c2ef070 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:39.758590 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc44064-1b68-4788-852c-4d8f078b4425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:39.759278 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc44064-1b68-4788-852c-4d8f078b4425 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:39.759601 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc44064-1b68-4788-852c-4d8f078b4425 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:39.760146 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc44064-1b68-4788-852c-4d8f078b4425 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:39.760876 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc44064-1b68-4788-852c-4d8f078b4425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:39.762523 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc44064-1b68-4788-852c-4d8f078b4425 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:39.770670 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f1a86-446b-4d58-a8cb-5aaa7c2ef070 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:39.776254 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1324/2656] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:39.778302 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f0b90-0bb4-4ff4-bbbe-cac93c5e4af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:39.779732 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f0b90-0bb4-4ff4-bbbe-cac93c5e4af9 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:39.780501 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f0b90-0bb4-4ff4-bbbe-cac93c5e4af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:39.780917 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc44064-1b68-4788-852c-4d8f078b4425 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:39.782626 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f0b90-0bb4-4ff4-bbbe-cac93c5e4af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:39.785189 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283f0b90-0bb4-4ff4-bbbe-cac93c5e4af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:39.791958 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc44064-1b68-4788-852c-4d8f078b4425 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:39.799905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283f0b90-0bb4-4ff4-bbbe-cac93c5e4af9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:39.809066 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f05b6-a650-4743-9837-a85d29ce8d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:39.809977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f05b6-a650-4743-9837-a85d29ce8d45 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:39.810284 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f05b6-a650-4743-9837-a85d29ce8d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:39.810700 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f05b6-a650-4743-9837-a85d29ce8d45 admin admin] PATH_INFO: `/v3/projects/a2b960dd5dba4e27b0c1be8b2eae3a92/users/6ab450dd3d1d42e58579d2dea4e1169c/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:39.812651 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1333/2657] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:04:39.816970 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f0b90-0bb4-4ff4-bbbe-cac93c5e4af9 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:39.832046 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f05b6-a650-4743-9837-a85d29ce8d45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2b960dd5dba4e27b0c1be8b2eae3a92, user_id=6ab450dd3d1d42e58579d2dea4e1169c, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:39.834852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f05b6-a650-4743-9837-a85d29ce8d45 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:39.836401 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:39.840570 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f0b90-0bb4-4ff4-bbbe-cac93c5e4af9 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:39.848998 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81b75d-9c5e-4cab-80b1-bd92572995a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:39.850573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81b75d-9c5e-4cab-80b1-bd92572995a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:39.851085 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81b75d-9c5e-4cab-80b1-bd92572995a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:39.851472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81b75d-9c5e-4cab-80b1-bd92572995a7 admin admin] PATH_INFO: `/v3/projects/9136a7cf76b943e796eebc82d60b7ae7/users/c8d4821bc35c4fe4bb3566010e803706/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:39.853060 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1325/2658] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:39 2022] PUT /identity/v3/projects/a2b960dd5dba4e27b0c1be8b2eae3a92/users/6ab450dd3d1d42e58579d2dea4e1169c/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:39.868662 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fffae1-5614-4424-a798-ccfa91e0e147 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:39.868662 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fffae1-5614-4424-a798-ccfa91e0e147 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:39.869034 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fffae1-5614-4424-a798-ccfa91e0e147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:39.885316 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81b75d-9c5e-4cab-80b1-bd92572995a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9136a7cf76b943e796eebc82d60b7ae7, user_id=c8d4821bc35c4fe4bb3566010e803706, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:39.888117 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81b75d-9c5e-4cab-80b1-bd92572995a7 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:39.912881 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1334/2659] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:39 2022] PUT /identity/v3/projects/9136a7cf76b943e796eebc82d60b7ae7/users/c8d4821bc35c4fe4bb3566010e803706/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:04:39.916348 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-15fffae1-5614-4424-a798-ccfa91e0e147 None None] MFA Rules not processed for user `6ab450dd3d1d42e58579d2dea4e1169c`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:39.917504 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81b75d-9c5e-4cab-80b1-bd92572995a7 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:39.917872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-15fffae1-5614-4424-a798-ccfa91e0e147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:39.999854 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81b75d-9c5e-4cab-80b1-bd92572995a7 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:40.006326 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:40.014280 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0bef48-5179-4d28-bea0-5921f0352435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:40.019682 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bef48-5179-4d28-bea0-5921f0352435 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:40.024262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1326/2660] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:04:40.024441 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bef48-5179-4d28-bea0-5921f0352435 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:40.024441 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bef48-5179-4d28-bea0-5921f0352435 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:40.024441 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0bef48-5179-4d28-bea0-5921f0352435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:40.034328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fffae1-5614-4424-a798-ccfa91e0e147 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:40.036590 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0bef48-5179-4d28-bea0-5921f0352435 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:40.037186 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-15fffae1-5614-4424-a798-ccfa91e0e147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:40.060654 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1335/2661] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:04:40.065430 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0bef48-5179-4d28-bea0-5921f0352435 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:40.076025 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0bef48-5179-4d28-bea0-5921f0352435 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:40.080878 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a68e3-30a1-4b71-b6e0-c8e7b7175e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:40.081881 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a68e3-30a1-4b71-b6e0-c8e7b7175e0c admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:40.082070 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a68e3-30a1-4b71-b6e0-c8e7b7175e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:40.082341 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a68e3-30a1-4b71-b6e0-c8e7b7175e0c admin admin] PATH_INFO: `/v3/projects/9136a7cf76b943e796eebc82d60b7ae7/users/c8d4821bc35c4fe4bb3566010e803706/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:40.084481 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a68e3-30a1-4b71-b6e0-c8e7b7175e0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9136a7cf76b943e796eebc82d60b7ae7, user_id=c8d4821bc35c4fe4bb3566010e803706, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:40.087196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a68e3-30a1-4b71-b6e0-c8e7b7175e0c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:40.105718 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1336/2662] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:40 2022] PUT /identity/v3/projects/9136a7cf76b943e796eebc82d60b7ae7/users/c8d4821bc35c4fe4bb3566010e803706/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:40.116488 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fffae1-5614-4424-a798-ccfa91e0e147 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:40.117813 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5e421-5296-4bed-81bd-a7feea4efe40 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:40.118157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5e421-5296-4bed-81bd-a7feea4efe40 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:40.118448 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5e421-5296-4bed-81bd-a7feea4efe40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:40.153664 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-bfb5e421-5296-4bed-81bd-a7feea4efe40 None None] MFA Rules not processed for user `c8d4821bc35c4fe4bb3566010e803706`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:40.154988 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-bfb5e421-5296-4bed-81bd-a7feea4efe40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:40.196214 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864b4d6-4e10-41a0-af11-f6f8fee849d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:40.197253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864b4d6-4e10-41a0-af11-f6f8fee849d7 service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:40.197727 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864b4d6-4e10-41a0-af11-f6f8fee849d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:40.198221 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864b4d6-4e10-41a0-af11-f6f8fee849d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:40.268254 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1337/2663] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:04:40.281568 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0864b4d6-4e10-41a0-af11-f6f8fee849d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:40.283076 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5e421-5296-4bed-81bd-a7feea4efe40 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:40.283226 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0864b4d6-4e10-41a0-af11-f6f8fee849d7 service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:40.284993 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-bfb5e421-5296-4bed-81bd-a7feea4efe40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:40.293233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1327/2664] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:40.300167 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5e421-5296-4bed-81bd-a7feea4efe40 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:40.304480 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56444c-0c84-469d-bee7-db1dd408ae35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:40.305438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56444c-0c84-469d-bee7-db1dd408ae35 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:40.305959 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56444c-0c84-469d-bee7-db1dd408ae35 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:40.306532 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56444c-0c84-469d-bee7-db1dd408ae35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:40.331409 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56444c-0c84-469d-bee7-db1dd408ae35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:40.333202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56444c-0c84-469d-bee7-db1dd408ae35 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:40.359395 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1338/2665] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:41.146831 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864b4d6-4e10-41a0-af11-f6f8fee849d7 service cinder] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:41.225838 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864b4d6-4e10-41a0-af11-f6f8fee849d7 service cinder] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:41.230648 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fa127-eadf-4c8f-b0df-4bb60fda1563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:41.231634 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fa127-eadf-4c8f-b0df-4bb60fda1563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:41.231990 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fa127-eadf-4c8f-b0df-4bb60fda1563 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:41.232335 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fa127-eadf-4c8f-b0df-4bb60fda1563 admin admin] PATH_INFO: `/v3/users/64a6d267d37c4eabaa6053c3dd2fd158` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:41.233767 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fa127-eadf-4c8f-b0df-4bb60fda1563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64a6d267d37c4eabaa6053c3dd2fd158)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:41.235982 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fa127-eadf-4c8f-b0df-4bb60fda1563 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:41.242400 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=100 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:41.301631 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-341fa127-eadf-4c8f-b0df-4bb60fda1563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64a6d267d37c4eabaa6053c3dd2fd158'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:41.319127 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-341fa127-eadf-4c8f-b0df-4bb60fda1563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64a6d267d37c4eabaa6053c3dd2fd158'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:41.327578 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-341fa127-eadf-4c8f-b0df-4bb60fda1563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64a6d267d37c4eabaa6053c3dd2fd158'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:41.328467 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-341fa127-eadf-4c8f-b0df-4bb60fda1563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64a6d267d37c4eabaa6053c3dd2fd158'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:41.329120 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-341fa127-eadf-4c8f-b0df-4bb60fda1563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64a6d267d37c4eabaa6053c3dd2fd158'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:41.347950 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1328/2666] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:04:41 2022] DELETE /identity/v3/users/64a6d267d37c4eabaa6053c3dd2fd158 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:41.353496 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedad1c-556a-4b05-a488-8cb6f1de8161 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:41.354545 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedad1c-556a-4b05-a488-8cb6f1de8161 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:41.354545 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedad1c-556a-4b05-a488-8cb6f1de8161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:41.375018 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-7aedad1c-556a-4b05-a488-8cb6f1de8161 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:41.376356 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7aedad1c-556a-4b05-a488-8cb6f1de8161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:41.416896 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:41.468980 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1339/2667] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:41.481141 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fa127-eadf-4c8f-b0df-4bb60fda1563 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:41.482760 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-341fa127-eadf-4c8f-b0df-4bb60fda1563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:41.562553 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fa127-eadf-4c8f-b0df-4bb60fda1563 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:41.568299 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244bef08-218b-40bb-b6ef-aac2cb91b371 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:41.568964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244bef08-218b-40bb-b6ef-aac2cb91b371 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:41.569247 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244bef08-218b-40bb-b6ef-aac2cb91b371 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:41.569522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244bef08-218b-40bb-b6ef-aac2cb91b371 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:41.583375 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244bef08-218b-40bb-b6ef-aac2cb91b371 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:41.584849 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244bef08-218b-40bb-b6ef-aac2cb91b371 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:41.596635 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1329/2668] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:04:42.029628 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aedad1c-556a-4b05-a488-8cb6f1de8161 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:42.032915 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7aedad1c-556a-4b05-a488-8cb6f1de8161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:42.073900 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aedad1c-556a-4b05-a488-8cb6f1de8161 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:42.096605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e2392-96a7-483a-857a-f21755049321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:42.098387 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e2392-96a7-483a-857a-f21755049321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:42.098869 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e2392-96a7-483a-857a-f21755049321 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:42.099388 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e2392-96a7-483a-857a-f21755049321 admin admin] PATH_INFO: `/v3/projects/4bac591af3764a5c9747f8236ad98cf7` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:42.101013 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e2392-96a7-483a-857a-f21755049321 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bac591af3764a5c9747f8236ad98cf7)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:42.103854 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e2392-96a7-483a-857a-f21755049321 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:42.191619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-682e2392-96a7-483a-857a-f21755049321 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bac591af3764a5c9747f8236ad98cf7', 'request_id': 'req-682e2392-96a7-483a-857a-f21755049321'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:42.199747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-682e2392-96a7-483a-857a-f21755049321 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bac591af3764a5c9747f8236ad98cf7', 'request_id': 'req-682e2392-96a7-483a-857a-f21755049321'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:42.206670 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-682e2392-96a7-483a-857a-f21755049321 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bac591af3764a5c9747f8236ad98cf7', 'request_id': 'req-682e2392-96a7-483a-857a-f21755049321'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:42.229964 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-682e2392-96a7-483a-857a-f21755049321 admin admin] The token cache is being invalidate because project 4bac591af3764a5c9747f8236ad98cf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:04:42.230688 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-682e2392-96a7-483a-857a-f21755049321 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:42.231477 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-682e2392-96a7-483a-857a-f21755049321 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:42.234736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1340/2669] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:04:42 2022] DELETE /identity/v3/projects/4bac591af3764a5c9747f8236ad98cf7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:42.239186 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244bef08-218b-40bb-b6ef-aac2cb91b371 service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:42.272415 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244bef08-218b-40bb-b6ef-aac2cb91b371 service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:42.348255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727d22a-2f63-47dc-94c1-a2707fe8642a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:42.349833 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727d22a-2f63-47dc-94c1-a2707fe8642a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:42.350270 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727d22a-2f63-47dc-94c1-a2707fe8642a admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:42.350556 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727d22a-2f63-47dc-94c1-a2707fe8642a admin admin] PATH_INFO: `/v3/users/c8d4821bc35c4fe4bb3566010e803706` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:42.352244 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3727d22a-2f63-47dc-94c1-a2707fe8642a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8d4821bc35c4fe4bb3566010e803706)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:42.354631 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3727d22a-2f63-47dc-94c1-a2707fe8642a admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:42.389140 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:42.414966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3727d22a-2f63-47dc-94c1-a2707fe8642a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8d4821bc35c4fe4bb3566010e803706'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:42.431797 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3727d22a-2f63-47dc-94c1-a2707fe8642a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8d4821bc35c4fe4bb3566010e803706'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:42.442147 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3727d22a-2f63-47dc-94c1-a2707fe8642a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8d4821bc35c4fe4bb3566010e803706'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:42.442829 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3727d22a-2f63-47dc-94c1-a2707fe8642a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8d4821bc35c4fe4bb3566010e803706'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:42.443521 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-3727d22a-2f63-47dc-94c1-a2707fe8642a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8d4821bc35c4fe4bb3566010e803706'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:42.462348 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1330/2670] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:04:42 2022] DELETE /identity/v3/users/c8d4821bc35c4fe4bb3566010e803706 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:42.786801 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e2392-96a7-483a-857a-f21755049321 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:42.788711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-682e2392-96a7-483a-857a-f21755049321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:42.806093 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e2392-96a7-483a-857a-f21755049321 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:42.916153 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9898e-0e0d-47d9-9943-509c5832bd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:42.917405 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9898e-0e0d-47d9-9943-509c5832bd80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:42.917987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9898e-0e0d-47d9-9943-509c5832bd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:42.918534 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9898e-0e0d-47d9-9943-509c5832bd80 admin admin] PATH_INFO: `/v3/projects/9136a7cf76b943e796eebc82d60b7ae7` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:42.920015 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca9898e-0e0d-47d9-9943-509c5832bd80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9136a7cf76b943e796eebc82d60b7ae7)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:42.922663 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca9898e-0e0d-47d9-9943-509c5832bd80 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:43.076153 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2ca9898e-0e0d-47d9-9943-509c5832bd80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9136a7cf76b943e796eebc82d60b7ae7', 'request_id': 'req-2ca9898e-0e0d-47d9-9943-509c5832bd80'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:43.089514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2ca9898e-0e0d-47d9-9943-509c5832bd80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9136a7cf76b943e796eebc82d60b7ae7', 'request_id': 'req-2ca9898e-0e0d-47d9-9943-509c5832bd80'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:43.093516 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:43.102064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2ca9898e-0e0d-47d9-9943-509c5832bd80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9136a7cf76b943e796eebc82d60b7ae7', 'request_id': 'req-2ca9898e-0e0d-47d9-9943-509c5832bd80'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:43.124935 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2ca9898e-0e0d-47d9-9943-509c5832bd80 admin admin] The token cache is being invalidate because project 9136a7cf76b943e796eebc82d60b7ae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:04:43.125331 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2ca9898e-0e0d-47d9-9943-509c5832bd80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:43.126179 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2ca9898e-0e0d-47d9-9943-509c5832bd80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:43.128198 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1341/2671] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:04:42 2022] DELETE /identity/v3/projects/9136a7cf76b943e796eebc82d60b7ae7 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:43.135191 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256d562-b3b2-4c24-b7dd-1bad75a78c22 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:43.135574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256d562-b3b2-4c24-b7dd-1bad75a78c22 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:43.135931 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256d562-b3b2-4c24-b7dd-1bad75a78c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:43.158465 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-6256d562-b3b2-4c24-b7dd-1bad75a78c22 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:43.159842 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-6256d562-b3b2-4c24-b7dd-1bad75a78c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:43.236645 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1331/2672] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:43.244218 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797531f-d6d0-4ada-9bfa-2b508b59a78e None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:43.244511 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797531f-d6d0-4ada-9bfa-2b508b59a78e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:43.244795 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797531f-d6d0-4ada-9bfa-2b508b59a78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:43.267060 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-5797531f-d6d0-4ada-9bfa-2b508b59a78e None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:43.268484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5797531f-d6d0-4ada-9bfa-2b508b59a78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:43.276988 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1342/2673] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:43.281585 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256d562-b3b2-4c24-b7dd-1bad75a78c22 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:43.293861 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256d562-b3b2-4c24-b7dd-1bad75a78c22 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:43.298749 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b15262-3ad1-4b97-8e69-740c1a4463ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:43.299514 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b15262-3ad1-4b97-8e69-740c1a4463ea admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:43.299869 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b15262-3ad1-4b97-8e69-740c1a4463ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:43.300286 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b15262-3ad1-4b97-8e69-740c1a4463ea admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:43.302926 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b15262-3ad1-4b97-8e69-740c1a4463ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:04:43.302926 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b15262-3ad1-4b97-8e69-740c1a4463ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:43.303254 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b15262-3ad1-4b97-8e69-740c1a4463ea admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:43.321325 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1332/2674] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 15:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:04:43.322161 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797531f-d6d0-4ada-9bfa-2b508b59a78e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:43.334057 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797531f-d6d0-4ada-9bfa-2b508b59a78e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:43.339405 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af7fc9-5829-4449-ac49-8f12979a9dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:43.340703 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af7fc9-5829-4449-ac49-8f12979a9dcf admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:43.340703 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af7fc9-5829-4449-ac49-8f12979a9dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:43.340703 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af7fc9-5829-4449-ac49-8f12979a9dcf admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:43.341511 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af7fc9-5829-4449-ac49-8f12979a9dcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:43.343365 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af7fc9-5829-4449-ac49-8f12979a9dcf admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:43.379096 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1343/2675] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:04:43 2022] POST /identity/v3/projects => generated 357 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:43.380877 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b15262-3ad1-4b97-8e69-740c1a4463ea admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:43.393631 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b15262-3ad1-4b97-8e69-740c1a4463ea admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:43.464541 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67e6d9-0fcc-4846-85b7-2342c21204a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:43.465461 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67e6d9-0fcc-4846-85b7-2342c21204a1 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:43.465821 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67e6d9-0fcc-4846-85b7-2342c21204a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:43.466194 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67e6d9-0fcc-4846-85b7-2342c21204a1 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:43.466953 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67e6d9-0fcc-4846-85b7-2342c21204a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:43.469165 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67e6d9-0fcc-4846-85b7-2342c21204a1 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:43.494185 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1333/2676] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:04:43 2022] POST /identity/v3/users => generated 279 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:43.497373 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af7fc9-5829-4449-ac49-8f12979a9dcf admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:43.508766 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af7fc9-5829-4449-ac49-8f12979a9dcf admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:43.514512 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30ceee-ad31-4b3a-aaeb-fc69c5958de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:43.515237 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30ceee-ad31-4b3a-aaeb-fc69c5958de0 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:43.515520 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30ceee-ad31-4b3a-aaeb-fc69c5958de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:43.515744 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30ceee-ad31-4b3a-aaeb-fc69c5958de0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:43.517005 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30ceee-ad31-4b3a-aaeb-fc69c5958de0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:43.518650 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30ceee-ad31-4b3a-aaeb-fc69c5958de0 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:43.529902 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1344/2677] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:43.535162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67e6d9-0fcc-4846-85b7-2342c21204a1 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:43.546471 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67e6d9-0fcc-4846-85b7-2342c21204a1 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:43.551339 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f75b10-6bd5-493e-aa21-98fc78ef5eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:43.552328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f75b10-6bd5-493e-aa21-98fc78ef5eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:43.552817 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f75b10-6bd5-493e-aa21-98fc78ef5eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:43.553319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f75b10-6bd5-493e-aa21-98fc78ef5eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:43.554608 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f75b10-6bd5-493e-aa21-98fc78ef5eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:43.557725 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f75b10-6bd5-493e-aa21-98fc78ef5eb8 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:43.571255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1334/2678] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:43.575923 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30ceee-ad31-4b3a-aaeb-fc69c5958de0 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:43.588263 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30ceee-ad31-4b3a-aaeb-fc69c5958de0 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:43.593097 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a78025-92e0-4edd-ba67-16f82f97a07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:43.593982 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a78025-92e0-4edd-ba67-16f82f97a07c admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:43.594289 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a78025-92e0-4edd-ba67-16f82f97a07c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:43.594564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a78025-92e0-4edd-ba67-16f82f97a07c admin admin] PATH_INFO: `/v3/projects/b8bb8b786afe417ea46584106431cbbd/users/b863611fc5c94e61839793c49e21d775/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:43.611211 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a78025-92e0-4edd-ba67-16f82f97a07c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8bb8b786afe417ea46584106431cbbd, user_id=b863611fc5c94e61839793c49e21d775, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:43.615225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a78025-92e0-4edd-ba67-16f82f97a07c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:43.642064 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1345/2679] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:43 2022] PUT /identity/v3/projects/b8bb8b786afe417ea46584106431cbbd/users/b863611fc5c94e61839793c49e21d775/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:43.644662 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20913b68-f447-4ea1-8330-6ae6eee1472e None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:43.644662 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20913b68-f447-4ea1-8330-6ae6eee1472e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:43.645104 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20913b68-f447-4ea1-8330-6ae6eee1472e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:43.668292 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:43.678548 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-20913b68-f447-4ea1-8330-6ae6eee1472e None None] MFA Rules not processed for user `b863611fc5c94e61839793c49e21d775`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:43.681627 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-20913b68-f447-4ea1-8330-6ae6eee1472e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:43.770071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1335/2680] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:43 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:43.799720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a78025-92e0-4edd-ba67-16f82f97a07c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:43.802336 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c5a78025-92e0-4edd-ba67-16f82f97a07c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:43.866645 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a78025-92e0-4edd-ba67-16f82f97a07c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:43.871219 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb3056f-20d1-4487-a6ca-8beaf18f3c16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:43.871876 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3056f-20d1-4487-a6ca-8beaf18f3c16 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:43.872162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3056f-20d1-4487-a6ca-8beaf18f3c16 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:43.872454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3056f-20d1-4487-a6ca-8beaf18f3c16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:43.883817 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb3056f-20d1-4487-a6ca-8beaf18f3c16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:43.885544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb3056f-20d1-4487-a6ca-8beaf18f3c16 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:43.896603 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1346/2681] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:43 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:43.911082 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20913b68-f447-4ea1-8330-6ae6eee1472e None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:43.986417 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20913b68-f447-4ea1-8330-6ae6eee1472e None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:43.991359 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b09e84-3c82-4e6a-a93d-afce68f746cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:43.992371 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b09e84-3c82-4e6a-a93d-afce68f746cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:43.992728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b09e84-3c82-4e6a-a93d-afce68f746cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:43.993105 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b09e84-3c82-4e6a-a93d-afce68f746cb admin admin] PATH_INFO: `/v3/users/b863611fc5c94e61839793c49e21d775` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:43.994659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b09e84-3c82-4e6a-a93d-afce68f746cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b863611fc5c94e61839793c49e21d775)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:43.996725 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b09e84-3c82-4e6a-a93d-afce68f746cb admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:44.057829 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-76b09e84-3c82-4e6a-a93d-afce68f746cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b863611fc5c94e61839793c49e21d775'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:44.074753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-76b09e84-3c82-4e6a-a93d-afce68f746cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b863611fc5c94e61839793c49e21d775'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:44.086237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-76b09e84-3c82-4e6a-a93d-afce68f746cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b863611fc5c94e61839793c49e21d775'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:44.088471 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-76b09e84-3c82-4e6a-a93d-afce68f746cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b863611fc5c94e61839793c49e21d775'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:44.089312 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-76b09e84-3c82-4e6a-a93d-afce68f746cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b863611fc5c94e61839793c49e21d775'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:44.110058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1336/2682] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:04:43 2022] DELETE /identity/v3/users/b863611fc5c94e61839793c49e21d775 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:44.112956 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa456b-89ca-4888-9fa6-606ce6672446 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:44.113349 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa456b-89ca-4888-9fa6-606ce6672446 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:44.113714 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa456b-89ca-4888-9fa6-606ce6672446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:44.133577 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-cbfa456b-89ca-4888-9fa6-606ce6672446 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:44.135131 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-cbfa456b-89ca-4888-9fa6-606ce6672446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:44.228353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1347/2683] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:44.240068 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b09e84-3c82-4e6a-a93d-afce68f746cb admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:44.242034 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-76b09e84-3c82-4e6a-a93d-afce68f746cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:44.282591 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:44.340004 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b09e84-3c82-4e6a-a93d-afce68f746cb admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:44.344538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fbfb5-c5ba-4d57-b775-ea15ec8d6805 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:44.345208 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fbfb5-c5ba-4d57-b775-ea15ec8d6805 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:44.345490 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fbfb5-c5ba-4d57-b775-ea15ec8d6805 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:44.345767 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fbfb5-c5ba-4d57-b775-ea15ec8d6805 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:44.360198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195fbfb5-c5ba-4d57-b775-ea15ec8d6805 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:44.361758 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195fbfb5-c5ba-4d57-b775-ea15ec8d6805 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:44.376522 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1337/2684] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:44.687833 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa456b-89ca-4888-9fa6-606ce6672446 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:44.689766 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-cbfa456b-89ca-4888-9fa6-606ce6672446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:44.706231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa456b-89ca-4888-9fa6-606ce6672446 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:44.724382 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c38af-7321-47ce-b6f5-411906a774d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:44.725577 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c38af-7321-47ce-b6f5-411906a774d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:44.725990 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c38af-7321-47ce-b6f5-411906a774d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:44.726476 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c38af-7321-47ce-b6f5-411906a774d7 admin admin] PATH_INFO: `/v3/projects/b8bb8b786afe417ea46584106431cbbd` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:44.727987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c38af-7321-47ce-b6f5-411906a774d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8bb8b786afe417ea46584106431cbbd)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:44.730075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c38af-7321-47ce-b6f5-411906a774d7 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:44.761364 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:44.790238 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-185c38af-7321-47ce-b6f5-411906a774d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8bb8b786afe417ea46584106431cbbd', 'request_id': 'req-185c38af-7321-47ce-b6f5-411906a774d7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:44.796427 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-185c38af-7321-47ce-b6f5-411906a774d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8bb8b786afe417ea46584106431cbbd', 'request_id': 'req-185c38af-7321-47ce-b6f5-411906a774d7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:44.804520 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-185c38af-7321-47ce-b6f5-411906a774d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8bb8b786afe417ea46584106431cbbd', 'request_id': 'req-185c38af-7321-47ce-b6f5-411906a774d7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:44.819780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-185c38af-7321-47ce-b6f5-411906a774d7 admin admin] The token cache is being invalidate because project b8bb8b786afe417ea46584106431cbbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:04:44.820074 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-185c38af-7321-47ce-b6f5-411906a774d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:44.821086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-185c38af-7321-47ce-b6f5-411906a774d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:44.823261 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1348/2685] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:04:44 2022] DELETE /identity/v3/projects/b8bb8b786afe417ea46584106431cbbd => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:44.828712 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ab597-ddc2-463b-a25e-a1278d131061 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:44.829005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ab597-ddc2-463b-a25e-a1278d131061 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:44.829338 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ab597-ddc2-463b-a25e-a1278d131061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:44.850846 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-c14ab597-ddc2-463b-a25e-a1278d131061 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:44.852072 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-c14ab597-ddc2-463b-a25e-a1278d131061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:44.937992 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1338/2686] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:04:44.946646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47841995-c3fc-4a32-89f5-12333a3361ef None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:44.946940 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47841995-c3fc-4a32-89f5-12333a3361ef None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:44.947223 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47841995-c3fc-4a32-89f5-12333a3361ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:44.968425 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-47841995-c3fc-4a32-89f5-12333a3361ef None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:44.969749 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-47841995-c3fc-4a32-89f5-12333a3361ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:44.980626 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1349/2687] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:44.984883 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ab597-ddc2-463b-a25e-a1278d131061 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:44.997542 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ab597-ddc2-463b-a25e-a1278d131061 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:45.003859 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ebdd6-14a3-4df4-ad29-08e45c33d650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:45.004530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ebdd6-14a3-4df4-ad29-08e45c33d650 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:45.004813 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ebdd6-14a3-4df4-ad29-08e45c33d650 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:45.005092 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ebdd6-14a3-4df4-ad29-08e45c33d650 admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:45.005827 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ebdd6-14a3-4df4-ad29-08e45c33d650 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:04:45.006201 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ebdd6-14a3-4df4-ad29-08e45c33d650 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:45.007822 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ebdd6-14a3-4df4-ad29-08e45c33d650 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:45.025629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1339/2688] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 15:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:04:45.030395 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47841995-c3fc-4a32-89f5-12333a3361ef None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:45.046833 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47841995-c3fc-4a32-89f5-12333a3361ef None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:45.051911 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0d174-e9b8-44b9-b9a8-0d0dafad069a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:45.052601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0d174-e9b8-44b9-b9a8-0d0dafad069a admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:45.052891 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0d174-e9b8-44b9-b9a8-0d0dafad069a admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:45.053170 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0d174-e9b8-44b9-b9a8-0d0dafad069a admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:45.053887 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0d174-e9b8-44b9-b9a8-0d0dafad069a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:45.055875 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0d174-e9b8-44b9-b9a8-0d0dafad069a admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:45.094129 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1350/2689] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:04:45 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:04:45.098842 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ebdd6-14a3-4df4-ad29-08e45c33d650 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:45.110451 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ebdd6-14a3-4df4-ad29-08e45c33d650 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:45.180133 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8193e1-f9b1-4731-8a77-cd4baa05feb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:45.180881 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8193e1-f9b1-4731-8a77-cd4baa05feb0 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:45.181224 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8193e1-f9b1-4731-8a77-cd4baa05feb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:45.181574 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8193e1-f9b1-4731-8a77-cd4baa05feb0 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:45.182319 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8193e1-f9b1-4731-8a77-cd4baa05feb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:45.184307 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8193e1-f9b1-4731-8a77-cd4baa05feb0 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:45.216827 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1340/2690] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:04:45 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:45.220674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0d174-e9b8-44b9-b9a8-0d0dafad069a admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:45.234208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0d174-e9b8-44b9-b9a8-0d0dafad069a admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:45.239159 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1663e7-afb5-47a1-86a2-3ab0bd8c3de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:45.239838 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1663e7-afb5-47a1-86a2-3ab0bd8c3de4 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:45.240122 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1663e7-afb5-47a1-86a2-3ab0bd8c3de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:45.240402 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1663e7-afb5-47a1-86a2-3ab0bd8c3de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:45.241134 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1663e7-afb5-47a1-86a2-3ab0bd8c3de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:45.242811 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1663e7-afb5-47a1-86a2-3ab0bd8c3de4 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:45.261835 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1351/2691] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:45.265885 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8193e1-f9b1-4731-8a77-cd4baa05feb0 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:45.278702 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8193e1-f9b1-4731-8a77-cd4baa05feb0 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:45.283744 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12565ea6-0dba-4d8c-9835-ddfea65e2d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:45.284431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12565ea6-0dba-4d8c-9835-ddfea65e2d5b admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:45.284677 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12565ea6-0dba-4d8c-9835-ddfea65e2d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:45.284946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12565ea6-0dba-4d8c-9835-ddfea65e2d5b admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:45.285664 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12565ea6-0dba-4d8c-9835-ddfea65e2d5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:45.287662 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12565ea6-0dba-4d8c-9835-ddfea65e2d5b admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:45.303451 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1341/2692] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:45.307740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1663e7-afb5-47a1-86a2-3ab0bd8c3de4 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:45.319754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1663e7-afb5-47a1-86a2-3ab0bd8c3de4 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:45.325758 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b283d-f44e-45b5-8bca-89469f85a00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:45.326731 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b283d-f44e-45b5-8bca-89469f85a00e admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:45.327017 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b283d-f44e-45b5-8bca-89469f85a00e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:45.327297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b283d-f44e-45b5-8bca-89469f85a00e admin admin] PATH_INFO: `/v3/projects/f7cf9a15d7dc4113978e6e9ba84a6607/users/806d4bcb402a4fb1a762d86faf505fd4/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:45.349093 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b283d-f44e-45b5-8bca-89469f85a00e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7cf9a15d7dc4113978e6e9ba84a6607, user_id=806d4bcb402a4fb1a762d86faf505fd4, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:45.356344 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b283d-f44e-45b5-8bca-89469f85a00e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:45.388187 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1352/2693] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:45 2022] PUT /identity/v3/projects/f7cf9a15d7dc4113978e6e9ba84a6607/users/806d4bcb402a4fb1a762d86faf505fd4/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:04:45.390218 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba892036-bef9-46d1-99e4-96ea520658ca None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:45.391043 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba892036-bef9-46d1-99e4-96ea520658ca None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:45.391610 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba892036-bef9-46d1-99e4-96ea520658ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:45.434618 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-ba892036-bef9-46d1-99e4-96ea520658ca None None] MFA Rules not processed for user `806d4bcb402a4fb1a762d86faf505fd4`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:45.436468 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-ba892036-bef9-46d1-99e4-96ea520658ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:45.460570 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:45.533985 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1342/2694] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:45.541474 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b283d-f44e-45b5-8bca-89469f85a00e admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:45.692276 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b283d-f44e-45b5-8bca-89469f85a00e admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:45.704943 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c79bed2-6100-4c50-803c-e9f0e2626f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:45.705715 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79bed2-6100-4c50-803c-e9f0e2626f76 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:45.706093 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79bed2-6100-4c50-803c-e9f0e2626f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:45.706436 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79bed2-6100-4c50-803c-e9f0e2626f76 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:45.707196 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c79bed2-6100-4c50-803c-e9f0e2626f76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:45.709198 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c79bed2-6100-4c50-803c-e9f0e2626f76 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:45.743799 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1353/2695] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:04:45 2022] POST /identity/v3/projects => generated 373 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:45.746825 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba892036-bef9-46d1-99e4-96ea520658ca None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:45.761755 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba892036-bef9-46d1-99e4-96ea520658ca None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:45.840359 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70b534-ffb4-4df8-97a7-a2cd229a46f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:45.841058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70b534-ffb4-4df8-97a7-a2cd229a46f9 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:45.841344 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70b534-ffb4-4df8-97a7-a2cd229a46f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:45.841654 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70b534-ffb4-4df8-97a7-a2cd229a46f9 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:45.842530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70b534-ffb4-4df8-97a7-a2cd229a46f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:45.844831 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70b534-ffb4-4df8-97a7-a2cd229a46f9 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:45.874056 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1343/2696] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:04:45 2022] POST /identity/v3/users => generated 293 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:04:45.876210 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c79bed2-6100-4c50-803c-e9f0e2626f76 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:45.894166 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c79bed2-6100-4c50-803c-e9f0e2626f76 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:45.909767 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0016b772-a130-4ec9-973b-7527bf812385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:45.910509 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016b772-a130-4ec9-973b-7527bf812385 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:45.910798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016b772-a130-4ec9-973b-7527bf812385 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:45.911103 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016b772-a130-4ec9-973b-7527bf812385 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:45.912162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0016b772-a130-4ec9-973b-7527bf812385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:45.913547 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0016b772-a130-4ec9-973b-7527bf812385 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:45.928272 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1354/2697] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:04:45.932977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70b534-ffb4-4df8-97a7-a2cd229a46f9 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:45.945622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70b534-ffb4-4df8-97a7-a2cd229a46f9 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:45.953431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f998f4-b94a-423f-9921-20548576f69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:45.954664 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f998f4-b94a-423f-9921-20548576f69e admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:45.954947 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f998f4-b94a-423f-9921-20548576f69e admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:45.955225 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f998f4-b94a-423f-9921-20548576f69e admin admin] PATH_INFO: `/v3/projects/315be92bb5fd45debedfcbc80b09485a/users/00e2076dd6784b469a100a3829980ad6/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:45.990809 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f998f4-b94a-423f-9921-20548576f69e admin admin] RBAC: Authorizing `identity:create_grant(project_id=315be92bb5fd45debedfcbc80b09485a, user_id=00e2076dd6784b469a100a3829980ad6, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:45.993601 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f998f4-b94a-423f-9921-20548576f69e admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:46.011200 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1344/2698] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:45 2022] PUT /identity/v3/projects/315be92bb5fd45debedfcbc80b09485a/users/00e2076dd6784b469a100a3829980ad6/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:46.019521 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0016b772-a130-4ec9-973b-7527bf812385 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:46.039384 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0016b772-a130-4ec9-973b-7527bf812385 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:46.066555 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:46.114114 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd90df6-08ae-4052-b6a0-bcdf84a467e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:46.114114 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd90df6-08ae-4052-b6a0-bcdf84a467e2 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:46.114114 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd90df6-08ae-4052-b6a0-bcdf84a467e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:46.114559 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd90df6-08ae-4052-b6a0-bcdf84a467e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:46.114993 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd90df6-08ae-4052-b6a0-bcdf84a467e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:46.116616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd90df6-08ae-4052-b6a0-bcdf84a467e2 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:46.146706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1355/2699] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:46.155077 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f998f4-b94a-423f-9921-20548576f69e admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:46.174092 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f998f4-b94a-423f-9921-20548576f69e admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:46.178967 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77cf88-fbf9-4e11-a632-34959571faa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:46.179891 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77cf88-fbf9-4e11-a632-34959571faa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:46.180175 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77cf88-fbf9-4e11-a632-34959571faa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:46.180447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77cf88-fbf9-4e11-a632-34959571faa5 admin admin] PATH_INFO: `/v3/projects/315be92bb5fd45debedfcbc80b09485a/users/00e2076dd6784b469a100a3829980ad6/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:46.182918 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77cf88-fbf9-4e11-a632-34959571faa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=315be92bb5fd45debedfcbc80b09485a, user_id=00e2076dd6784b469a100a3829980ad6, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:46.185773 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77cf88-fbf9-4e11-a632-34959571faa5 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:46.197435 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1345/2700] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:46 2022] PUT /identity/v3/projects/315be92bb5fd45debedfcbc80b09485a/users/00e2076dd6784b469a100a3829980ad6/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:46.208554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa0540-489f-49c8-8ac0-ebc473181090 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:46.208554 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa0540-489f-49c8-8ac0-ebc473181090 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:46.208876 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa0540-489f-49c8-8ac0-ebc473181090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:46.239348 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-18aa0540-489f-49c8-8ac0-ebc473181090 None None] MFA Rules not processed for user `00e2076dd6784b469a100a3829980ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:46.240685 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-18aa0540-489f-49c8-8ac0-ebc473181090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:46.324306 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1356/2701] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:46.336155 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77cf88-fbf9-4e11-a632-34959571faa5 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:46.337905 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-8c77cf88-fbf9-4e11-a632-34959571faa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:46.414498 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77cf88-fbf9-4e11-a632-34959571faa5 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:46.418359 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189f52a-0d21-41f6-a16c-9dca01359301 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:46.419014 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189f52a-0d21-41f6-a16c-9dca01359301 service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:46.419373 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189f52a-0d21-41f6-a16c-9dca01359301 service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:46.419664 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189f52a-0d21-41f6-a16c-9dca01359301 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:46.449404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:46.497988 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e189f52a-0d21-41f6-a16c-9dca01359301 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:46.499887 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e189f52a-0d21-41f6-a16c-9dca01359301 service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:46.511144 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1346/2702] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:47.939089 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa0540-489f-49c8-8ac0-ebc473181090 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:47.941090 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-18aa0540-489f-49c8-8ac0-ebc473181090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:48.043613 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa0540-489f-49c8-8ac0-ebc473181090 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:48.051186 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1263a59-c17c-4ab5-9df4-0c9be2d83ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:48.052826 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1263a59-c17c-4ab5-9df4-0c9be2d83ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:48.053117 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1263a59-c17c-4ab5-9df4-0c9be2d83ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:48.053403 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1263a59-c17c-4ab5-9df4-0c9be2d83ea9 admin admin] PATH_INFO: `/v3/users/6ab450dd3d1d42e58579d2dea4e1169c` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:48.054705 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1263a59-c17c-4ab5-9df4-0c9be2d83ea9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ab450dd3d1d42e58579d2dea4e1169c)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:48.058774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1263a59-c17c-4ab5-9df4-0c9be2d83ea9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:48.122437 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:48.124682 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f1263a59-c17c-4ab5-9df4-0c9be2d83ea9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ab450dd3d1d42e58579d2dea4e1169c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:48.139681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f1263a59-c17c-4ab5-9df4-0c9be2d83ea9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ab450dd3d1d42e58579d2dea4e1169c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:48.149640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f1263a59-c17c-4ab5-9df4-0c9be2d83ea9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ab450dd3d1d42e58579d2dea4e1169c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:48.150488 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f1263a59-c17c-4ab5-9df4-0c9be2d83ea9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ab450dd3d1d42e58579d2dea4e1169c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:48.151242 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f1263a59-c17c-4ab5-9df4-0c9be2d83ea9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ab450dd3d1d42e58579d2dea4e1169c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:48.170837 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1357/2703] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:04:47 2022] DELETE /identity/v3/users/6ab450dd3d1d42e58579d2dea4e1169c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:48.172358 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716694ce-21b9-43a8-9a53-fff360d29919 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:48.172824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716694ce-21b9-43a8-9a53-fff360d29919 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:48.172952 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716694ce-21b9-43a8-9a53-fff360d29919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:48.193700 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-716694ce-21b9-43a8-9a53-fff360d29919 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:48.194975 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-716694ce-21b9-43a8-9a53-fff360d29919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:48.268253 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1347/2704] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:04:48.278246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1263a59-c17c-4ab5-9df4-0c9be2d83ea9 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:48.280803 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f1263a59-c17c-4ab5-9df4-0c9be2d83ea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:48.340569 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1263a59-c17c-4ab5-9df4-0c9be2d83ea9 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:48.345114 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1d6f9-9172-4579-bb58-542939be2595 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:48.345781 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1d6f9-9172-4579-bb58-542939be2595 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:48.346098 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1d6f9-9172-4579-bb58-542939be2595 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:48.346385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1d6f9-9172-4579-bb58-542939be2595 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:48.356720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1d6f9-9172-4579-bb58-542939be2595 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:48.358404 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1d6f9-9172-4579-bb58-542939be2595 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:48.369394 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1358/2705] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:48.654968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716694ce-21b9-43a8-9a53-fff360d29919 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:48.656573 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-716694ce-21b9-43a8-9a53-fff360d29919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:48.668636 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716694ce-21b9-43a8-9a53-fff360d29919 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:48.673654 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556a006-4c00-48e9-9d8f-ce503f6397c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:48.675210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556a006-4c00-48e9-9d8f-ce503f6397c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:48.675508 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556a006-4c00-48e9-9d8f-ce503f6397c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:48.675789 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556a006-4c00-48e9-9d8f-ce503f6397c7 admin admin] PATH_INFO: `/v3/projects/a2b960dd5dba4e27b0c1be8b2eae3a92` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:48.677277 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556a006-4c00-48e9-9d8f-ce503f6397c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2b960dd5dba4e27b0c1be8b2eae3a92)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:48.679320 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556a006-4c00-48e9-9d8f-ce503f6397c7 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:48.728939 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-8556a006-4c00-48e9-9d8f-ce503f6397c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2b960dd5dba4e27b0c1be8b2eae3a92', 'request_id': 'req-8556a006-4c00-48e9-9d8f-ce503f6397c7'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:48.734462 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-8556a006-4c00-48e9-9d8f-ce503f6397c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2b960dd5dba4e27b0c1be8b2eae3a92', 'request_id': 'req-8556a006-4c00-48e9-9d8f-ce503f6397c7'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:48.740644 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-8556a006-4c00-48e9-9d8f-ce503f6397c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2b960dd5dba4e27b0c1be8b2eae3a92', 'request_id': 'req-8556a006-4c00-48e9-9d8f-ce503f6397c7'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:48.754142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-8556a006-4c00-48e9-9d8f-ce503f6397c7 admin admin] The token cache is being invalidate because project a2b960dd5dba4e27b0c1be8b2eae3a92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:04:48.754432 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-8556a006-4c00-48e9-9d8f-ce503f6397c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:48.755073 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-8556a006-4c00-48e9-9d8f-ce503f6397c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:48.756816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1348/2706] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:04:48 2022] DELETE /identity/v3/projects/a2b960dd5dba4e27b0c1be8b2eae3a92 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:48.762327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc1b21-8027-46ab-be26-ed6da0d083d4 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:48.762629 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc1b21-8027-46ab-be26-ed6da0d083d4 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:48.762917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc1b21-8027-46ab-be26-ed6da0d083d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:48.781800 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-f7cc1b21-8027-46ab-be26-ed6da0d083d4 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:48.783068 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f7cc1b21-8027-46ab-be26-ed6da0d083d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:48.847836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1359/2707] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:48.856394 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f91542-ada3-437c-9323-31d0dd63c2ce None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:48.856698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f91542-ada3-437c-9323-31d0dd63c2ce None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:48.856974 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f91542-ada3-437c-9323-31d0dd63c2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:48.876070 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-76f91542-ada3-437c-9323-31d0dd63c2ce None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:48.877361 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-76f91542-ada3-437c-9323-31d0dd63c2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:48.885625 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1349/2708] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:04:48.890636 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc1b21-8027-46ab-be26-ed6da0d083d4 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:48.900860 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc1b21-8027-46ab-be26-ed6da0d083d4 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:48.907649 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee29319-2f7d-4291-9a50-b00874975ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:48.908324 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee29319-2f7d-4291-9a50-b00874975ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:48.908608 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee29319-2f7d-4291-9a50-b00874975ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:48.908886 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee29319-2f7d-4291-9a50-b00874975ba4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:48.909621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee29319-2f7d-4291-9a50-b00874975ba4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:04:48.909933 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee29319-2f7d-4291-9a50-b00874975ba4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:48.911619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee29319-2f7d-4291-9a50-b00874975ba4 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:48.924315 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1360/2709] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 15:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:04:48.928746 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f91542-ada3-437c-9323-31d0dd63c2ce None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:48.939149 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f91542-ada3-437c-9323-31d0dd63c2ce None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:48.943874 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba9089-d3d1-4462-9888-4d95b0f7b3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:48.944521 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba9089-d3d1-4462-9888-4d95b0f7b3be admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:48.944800 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba9089-d3d1-4462-9888-4d95b0f7b3be admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:48.945071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba9089-d3d1-4462-9888-4d95b0f7b3be admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:48.945754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba9089-d3d1-4462-9888-4d95b0f7b3be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:48.947701 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba9089-d3d1-4462-9888-4d95b0f7b3be admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:48.972679 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1350/2710] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:04:48 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:48.977065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee29319-2f7d-4291-9a50-b00874975ba4 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:48.988653 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee29319-2f7d-4291-9a50-b00874975ba4 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:49.036643 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:49.047480 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfdd2eb-46d3-469a-a0ee-e4c36914724c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:49.048157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfdd2eb-46d3-469a-a0ee-e4c36914724c admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:49.048446 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfdd2eb-46d3-469a-a0ee-e4c36914724c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:49.048728 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfdd2eb-46d3-469a-a0ee-e4c36914724c admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:49.049416 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfdd2eb-46d3-469a-a0ee-e4c36914724c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:49.051380 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfdd2eb-46d3-469a-a0ee-e4c36914724c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:49.072300 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1361/2711] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:04:48 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:04:49.076591 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba9089-d3d1-4462-9888-4d95b0f7b3be admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:49.086760 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba9089-d3d1-4462-9888-4d95b0f7b3be admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:49.091880 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f184ebf-0ebc-4151-87fa-206a8053009d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:49.092550 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f184ebf-0ebc-4151-87fa-206a8053009d admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:49.093276 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f184ebf-0ebc-4151-87fa-206a8053009d admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:49.093276 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f184ebf-0ebc-4151-87fa-206a8053009d admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:49.093801 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f184ebf-0ebc-4151-87fa-206a8053009d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:49.095470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f184ebf-0ebc-4151-87fa-206a8053009d admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:49.106805 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1351/2712] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:49.111525 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfdd2eb-46d3-469a-a0ee-e4c36914724c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:49.122094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfdd2eb-46d3-469a-a0ee-e4c36914724c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:49.127928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8ad63-4e96-412b-b055-283a2ec89a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:49.128534 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8ad63-4e96-412b-b055-283a2ec89a16 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:49.128885 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8ad63-4e96-412b-b055-283a2ec89a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:49.129507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8ad63-4e96-412b-b055-283a2ec89a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:49.130281 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8ad63-4e96-412b-b055-283a2ec89a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:49.132082 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8ad63-4e96-412b-b055-283a2ec89a16 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:49.143143 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1362/2713] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:49.147397 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f184ebf-0ebc-4151-87fa-206a8053009d admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:49.158095 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f184ebf-0ebc-4151-87fa-206a8053009d admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:49.162949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebe233-5740-4e71-a807-7b7503403d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:49.163840 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebe233-5740-4e71-a807-7b7503403d7a admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:49.164119 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebe233-5740-4e71-a807-7b7503403d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:49.164393 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebe233-5740-4e71-a807-7b7503403d7a admin admin] PATH_INFO: `/v3/projects/ffb2008af6964bc5b2e94a630bfe78e5/users/9c1bf697f0fd4e2cbc8d52cec772e446/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:49.167762 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:49.182781 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ebe233-5740-4e71-a807-7b7503403d7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffb2008af6964bc5b2e94a630bfe78e5, user_id=9c1bf697f0fd4e2cbc8d52cec772e446, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:49.185482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ebe233-5740-4e71-a807-7b7503403d7a admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:49.200355 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1352/2714] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:49 2022] PUT /identity/v3/projects/ffb2008af6964bc5b2e94a630bfe78e5/users/9c1bf697f0fd4e2cbc8d52cec772e446/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:49.205170 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7ae88-0823-4af8-8ef2-5d592423779a None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:49.205464 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7ae88-0823-4af8-8ef2-5d592423779a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:49.205883 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7ae88-0823-4af8-8ef2-5d592423779a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:49.224586 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-4ac7ae88-0823-4af8-8ef2-5d592423779a None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:49.227024 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4ac7ae88-0823-4af8-8ef2-5d592423779a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:49.239601 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebe233-5740-4e71-a807-7b7503403d7a admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:49.242513 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-70ebe233-5740-4e71-a807-7b7503403d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:49.311904 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1363/2715] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:49.314002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebe233-5740-4e71-a807-7b7503403d7a admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:49.318444 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7ae88-0823-4af8-8ef2-5d592423779a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:49.318928 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e83f09-c931-40eb-b0fc-9419e697da9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:49.319808 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e83f09-c931-40eb-b0fc-9419e697da9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:49.320009 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4ac7ae88-0823-4af8-8ef2-5d592423779a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:49.320120 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e83f09-c931-40eb-b0fc-9419e697da9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:49.320356 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e83f09-c931-40eb-b0fc-9419e697da9c admin admin] PATH_INFO: `/v3/users/806d4bcb402a4fb1a762d86faf505fd4` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:49.321847 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e83f09-c931-40eb-b0fc-9419e697da9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=806d4bcb402a4fb1a762d86faf505fd4)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:49.323817 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e83f09-c931-40eb-b0fc-9419e697da9c admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:49.373363 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f4e83f09-c931-40eb-b0fc-9419e697da9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '806d4bcb402a4fb1a762d86faf505fd4'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:49.381983 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7ae88-0823-4af8-8ef2-5d592423779a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:49.385942 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f4e83f09-c931-40eb-b0fc-9419e697da9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '806d4bcb402a4fb1a762d86faf505fd4'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:49.386117 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec09754-7db9-48eb-bef4-2518a8527380 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:49.386689 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec09754-7db9-48eb-bef4-2518a8527380 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:49.386977 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec09754-7db9-48eb-bef4-2518a8527380 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:49.387256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec09754-7db9-48eb-bef4-2518a8527380 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:49.395139 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f4e83f09-c931-40eb-b0fc-9419e697da9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '806d4bcb402a4fb1a762d86faf505fd4'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:49.395786 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f4e83f09-c931-40eb-b0fc-9419e697da9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '806d4bcb402a4fb1a762d86faf505fd4'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:49.396407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f4e83f09-c931-40eb-b0fc-9419e697da9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '806d4bcb402a4fb1a762d86faf505fd4'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:49.397426 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec09754-7db9-48eb-bef4-2518a8527380 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:49.398919 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec09754-7db9-48eb-bef4-2518a8527380 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:49.409946 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1364/2716] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:04:49.410273 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1353/2717] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:04:49 2022] DELETE /identity/v3/users/806d4bcb402a4fb1a762d86faf505fd4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:04:49.415574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ba6e8-b958-4267-b734-6273587cdd30 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:49.415891 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ba6e8-b958-4267-b734-6273587cdd30 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:49.416137 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ba6e8-b958-4267-b734-6273587cdd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:49.437261 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-343ba6e8-b958-4267-b734-6273587cdd30 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:49.438822 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-343ba6e8-b958-4267-b734-6273587cdd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:49.510836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1365/2718] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:04:49.524448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e83f09-c931-40eb-b0fc-9419e697da9c admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:49.526104 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f4e83f09-c931-40eb-b0fc-9419e697da9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:49.588668 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e83f09-c931-40eb-b0fc-9419e697da9c admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:49.592897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcc064-dd84-430e-915a-61324f6a9f04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:49.593744 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcc064-dd84-430e-915a-61324f6a9f04 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:49.593978 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcc064-dd84-430e-915a-61324f6a9f04 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:49.594388 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcc064-dd84-430e-915a-61324f6a9f04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:49.604887 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcc064-dd84-430e-915a-61324f6a9f04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:49.606624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcc064-dd84-430e-915a-61324f6a9f04 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:49.618060 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1354/2719] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:50.155910 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ba6e8-b958-4267-b734-6273587cdd30 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:50.157621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-343ba6e8-b958-4267-b734-6273587cdd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:50.170028 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ba6e8-b958-4267-b734-6273587cdd30 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:50.174810 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57adc9-3f23-4093-9cf6-2196fadfcc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:50.175687 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57adc9-3f23-4093-9cf6-2196fadfcc57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:50.175973 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57adc9-3f23-4093-9cf6-2196fadfcc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:50.176274 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57adc9-3f23-4093-9cf6-2196fadfcc57 admin admin] PATH_INFO: `/v3/projects/f7cf9a15d7dc4113978e6e9ba84a6607` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:50.177945 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57adc9-3f23-4093-9cf6-2196fadfcc57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7cf9a15d7dc4113978e6e9ba84a6607)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:50.179677 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57adc9-3f23-4093-9cf6-2196fadfcc57 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:50.192309 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:50.209380 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcc064-dd84-430e-915a-61324f6a9f04 service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:50.212346 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-a5dcc064-dd84-430e-915a-61324f6a9f04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:50.235908 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:50.247854 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0e57adc9-3f23-4093-9cf6-2196fadfcc57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7cf9a15d7dc4113978e6e9ba84a6607', 'request_id': 'req-0e57adc9-3f23-4093-9cf6-2196fadfcc57'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:50.265066 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0e57adc9-3f23-4093-9cf6-2196fadfcc57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7cf9a15d7dc4113978e6e9ba84a6607', 'request_id': 'req-0e57adc9-3f23-4093-9cf6-2196fadfcc57'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:50.277512 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0e57adc9-3f23-4093-9cf6-2196fadfcc57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7cf9a15d7dc4113978e6e9ba84a6607', 'request_id': 'req-0e57adc9-3f23-4093-9cf6-2196fadfcc57'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:50.299131 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0e57adc9-3f23-4093-9cf6-2196fadfcc57 admin admin] The token cache is being invalidate because project f7cf9a15d7dc4113978e6e9ba84a6607 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:04:50.299490 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0e57adc9-3f23-4093-9cf6-2196fadfcc57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:50.300082 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0e57adc9-3f23-4093-9cf6-2196fadfcc57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:50.304580 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1366/2720] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:04:50 2022] DELETE /identity/v3/projects/f7cf9a15d7dc4113978e6e9ba84a6607 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:50.305652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57adc9-3f23-4093-9cf6-2196fadfcc57 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:50.307489 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-0e57adc9-3f23-4093-9cf6-2196fadfcc57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:50.338514 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcc064-dd84-430e-915a-61324f6a9f04 service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:50.421694 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57adc9-3f23-4093-9cf6-2196fadfcc57 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:50.424820 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d631194-3634-4836-ab0f-7ca6be00112b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63789c0a113447ff90933d447cf5bac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:50.426113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d631194-3634-4836-ab0f-7ca6be00112b service placement] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:50.426598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d631194-3634-4836-ab0f-7ca6be00112b service placement] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:50.427183 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d631194-3634-4836-ab0f-7ca6be00112b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:50.428455 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86634d2-571e-43a3-9046-8243d8a54d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:50.429409 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86634d2-571e-43a3-9046-8243d8a54d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:50.429814 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86634d2-571e-43a3-9046-8243d8a54d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:50.430218 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-1d631194-3634-4836-ab0f-7ca6be00112b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:50.430756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86634d2-571e-43a3-9046-8243d8a54d35 admin admin] PATH_INFO: `/v3/users/00e2076dd6784b469a100a3829980ad6` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:50.432270 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86634d2-571e-43a3-9046-8243d8a54d35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00e2076dd6784b469a100a3829980ad6)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:50.434621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86634d2-571e-43a3-9046-8243d8a54d35 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:50.452448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d631194-3634-4836-ab0f-7ca6be00112b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:50.456293 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d631194-3634-4836-ab0f-7ca6be00112b service placement] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:50.473287 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1355/2721] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:04:50.503209 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a86634d2-571e-43a3-9046-8243d8a54d35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00e2076dd6784b469a100a3829980ad6'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:50.520505 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a86634d2-571e-43a3-9046-8243d8a54d35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00e2076dd6784b469a100a3829980ad6'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:50.531157 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a86634d2-571e-43a3-9046-8243d8a54d35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00e2076dd6784b469a100a3829980ad6'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:50.531927 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a86634d2-571e-43a3-9046-8243d8a54d35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00e2076dd6784b469a100a3829980ad6'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:50.532959 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a86634d2-571e-43a3-9046-8243d8a54d35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00e2076dd6784b469a100a3829980ad6'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:50.555352 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1367/2722] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:04:50 2022] DELETE /identity/v3/users/00e2076dd6784b469a100a3829980ad6 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:04:50.910495 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d631194-3634-4836-ab0f-7ca6be00112b service placement] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:50.925689 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d631194-3634-4836-ab0f-7ca6be00112b service placement] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:50.996649 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e09e6-c9d9-4df1-be71-8d936a267651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:50.998952 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e09e6-c9d9-4df1-be71-8d936a267651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:50.999482 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e09e6-c9d9-4df1-be71-8d936a267651 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:50.999804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e09e6-c9d9-4df1-be71-8d936a267651 admin admin] PATH_INFO: `/v3/projects/315be92bb5fd45debedfcbc80b09485a` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:51.001261 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e09e6-c9d9-4df1-be71-8d936a267651 admin admin] RBAC: Authorizing `identity:delete_project(project_id=315be92bb5fd45debedfcbc80b09485a)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:51.003497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e09e6-c9d9-4df1-be71-8d936a267651 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:51.065113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:51.073349 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ab8e09e6-c9d9-4df1-be71-8d936a267651 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '315be92bb5fd45debedfcbc80b09485a', 'request_id': 'req-ab8e09e6-c9d9-4df1-be71-8d936a267651'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:51.082550 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ab8e09e6-c9d9-4df1-be71-8d936a267651 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '315be92bb5fd45debedfcbc80b09485a', 'request_id': 'req-ab8e09e6-c9d9-4df1-be71-8d936a267651'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:51.089923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ab8e09e6-c9d9-4df1-be71-8d936a267651 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '315be92bb5fd45debedfcbc80b09485a', 'request_id': 'req-ab8e09e6-c9d9-4df1-be71-8d936a267651'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:51.111939 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ab8e09e6-c9d9-4df1-be71-8d936a267651 admin admin] The token cache is being invalidate because project 315be92bb5fd45debedfcbc80b09485a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:04:51.112413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ab8e09e6-c9d9-4df1-be71-8d936a267651 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:51.113121 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-ab8e09e6-c9d9-4df1-be71-8d936a267651 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:04:51.114968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1356/2723] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:04:50 2022] DELETE /identity/v3/projects/315be92bb5fd45debedfcbc80b09485a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:51.120237 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfbfe7-649b-4e2d-8311-e923d4de50fc None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:51.120561 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfbfe7-649b-4e2d-8311-e923d4de50fc None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:51.120817 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfbfe7-649b-4e2d-8311-e923d4de50fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:51.155346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-cfcfbfe7-649b-4e2d-8311-e923d4de50fc None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:51.157928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-cfcfbfe7-649b-4e2d-8311-e923d4de50fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:51.177182 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:51.255385 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1368/2724] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:51.258448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e32a9-21da-4fec-851c-268e62977393 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:51.258448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e32a9-21da-4fec-851c-268e62977393 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:51.258448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e32a9-21da-4fec-851c-268e62977393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:51.305853 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-794e32a9-21da-4fec-851c-268e62977393 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:51.307764 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-794e32a9-21da-4fec-851c-268e62977393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:51.320455 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1357/2725] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:51.324710 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfbfe7-649b-4e2d-8311-e923d4de50fc None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:51.358663 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfbfe7-649b-4e2d-8311-e923d4de50fc None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:51.364713 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d51a3-f5b1-42ea-a9c4-67cbb898a64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:51.365484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d51a3-f5b1-42ea-a9c4-67cbb898a64c admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:51.365845 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d51a3-f5b1-42ea-a9c4-67cbb898a64c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:51.366878 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d51a3-f5b1-42ea-a9c4-67cbb898a64c admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:51.367644 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d51a3-f5b1-42ea-a9c4-67cbb898a64c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:04:51.368122 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d51a3-f5b1-42ea-a9c4-67cbb898a64c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:51.370045 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d51a3-f5b1-42ea-a9c4-67cbb898a64c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:51.385567 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1369/2726] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 15:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:04:51.390269 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e32a9-21da-4fec-851c-268e62977393 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:51.400462 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e32a9-21da-4fec-851c-268e62977393 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:51.407058 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f05fc-192b-482f-85e7-56aadd7cd108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:51.407822 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f05fc-192b-482f-85e7-56aadd7cd108 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:51.408149 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f05fc-192b-482f-85e7-56aadd7cd108 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:51.408406 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f05fc-192b-482f-85e7-56aadd7cd108 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:51.409448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f05fc-192b-482f-85e7-56aadd7cd108 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:51.411606 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f05fc-192b-482f-85e7-56aadd7cd108 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:51.447431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1358/2727] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:04:51 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:51.452296 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d51a3-f5b1-42ea-a9c4-67cbb898a64c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:51.463866 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d51a3-f5b1-42ea-a9c4-67cbb898a64c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:51.529468 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629da9db-4bfb-4894-b00e-05c042060f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:51.530971 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629da9db-4bfb-4894-b00e-05c042060f59 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:51.531262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629da9db-4bfb-4894-b00e-05c042060f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:51.531596 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629da9db-4bfb-4894-b00e-05c042060f59 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:51.533368 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629da9db-4bfb-4894-b00e-05c042060f59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:51.535428 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629da9db-4bfb-4894-b00e-05c042060f59 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:51.562693 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1370/2728] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:04:51 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:51.627950 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f05fc-192b-482f-85e7-56aadd7cd108 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:51.639753 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f05fc-192b-482f-85e7-56aadd7cd108 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:51.645414 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac66c4-d9ba-41e1-afc4-89e078e9a59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:51.646151 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac66c4-d9ba-41e1-afc4-89e078e9a59c admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:51.646431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac66c4-d9ba-41e1-afc4-89e078e9a59c admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:51.646708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac66c4-d9ba-41e1-afc4-89e078e9a59c admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:51.647436 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eac66c4-d9ba-41e1-afc4-89e078e9a59c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:51.649162 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eac66c4-d9ba-41e1-afc4-89e078e9a59c admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:51.666890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1359/2729] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:51.668166 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629da9db-4bfb-4894-b00e-05c042060f59 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:51.680813 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629da9db-4bfb-4894-b00e-05c042060f59 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:51.690434 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b13fdf-7775-4c3a-ad08-988f28038c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:51.691194 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b13fdf-7775-4c3a-ad08-988f28038c2a admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:51.691465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b13fdf-7775-4c3a-ad08-988f28038c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:51.691947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b13fdf-7775-4c3a-ad08-988f28038c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:51.692694 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b13fdf-7775-4c3a-ad08-988f28038c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:51.694405 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b13fdf-7775-4c3a-ad08-988f28038c2a admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:51.706095 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1371/2730] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:04:51.710484 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac66c4-d9ba-41e1-afc4-89e078e9a59c admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:51.721626 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac66c4-d9ba-41e1-afc4-89e078e9a59c admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:51.727952 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e879fa1d-1449-4c15-8ced-e69819be4d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:51.729165 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879fa1d-1449-4c15-8ced-e69819be4d2a admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:51.729448 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879fa1d-1449-4c15-8ced-e69819be4d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:51.729745 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879fa1d-1449-4c15-8ced-e69819be4d2a admin admin] PATH_INFO: `/v3/projects/5e9e55701e7c4fe180a21d1934298240/users/f892dfa98adb40198ce1296134d1263c/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:51.746412 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e879fa1d-1449-4c15-8ced-e69819be4d2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e9e55701e7c4fe180a21d1934298240, user_id=f892dfa98adb40198ce1296134d1263c, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:51.749025 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e879fa1d-1449-4c15-8ced-e69819be4d2a admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:51.763262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1360/2731] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:51 2022] PUT /identity/v3/projects/5e9e55701e7c4fe180a21d1934298240/users/f892dfa98adb40198ce1296134d1263c/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:51.768887 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65127826-3e9b-4eb6-a204-0f85abf4ae81 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:51.769331 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65127826-3e9b-4eb6-a204-0f85abf4ae81 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:51.769619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65127826-3e9b-4eb6-a204-0f85abf4ae81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:51.788310 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-65127826-3e9b-4eb6-a204-0f85abf4ae81 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:51.789741 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-65127826-3e9b-4eb6-a204-0f85abf4ae81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:51.860533 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1372/2732] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:51.870875 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e879fa1d-1449-4c15-8ced-e69819be4d2a admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:51.872779 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-e879fa1d-1449-4c15-8ced-e69819be4d2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:51.964089 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e879fa1d-1449-4c15-8ced-e69819be4d2a admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:51.969450 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30203df5-29a6-4874-a4b3-a310432368dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:51.970477 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30203df5-29a6-4874-a4b3-a310432368dd service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:51.970930 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30203df5-29a6-4874-a4b3-a310432368dd service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:51.971210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30203df5-29a6-4874-a4b3-a310432368dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:51.983143 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30203df5-29a6-4874-a4b3-a310432368dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:51.984616 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30203df5-29a6-4874-a4b3-a310432368dd service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:51.996071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1361/2733] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:53.267228 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e448f9-c5fb-4d51-8282-ab60ec254f46 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:53.267768 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e448f9-c5fb-4d51-8282-ab60ec254f46 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:53.267768 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e448f9-c5fb-4d51-8282-ab60ec254f46 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:53.269189 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1373/2734] 10.222.0.78 () {54 vars in 921 bytes} [Wed May 25 15:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:04:53.281795 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bfcb0-1034-40a4-83a7-d90baa81eddb None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:53.281795 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bfcb0-1034-40a4-83a7-d90baa81eddb None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:53.281795 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bfcb0-1034-40a4-83a7-d90baa81eddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:53.361188 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-7b1bfcb0-1034-40a4-83a7-d90baa81eddb None None] MFA Rules not processed for user `e34e1d7ffbac49b0bde45829af0c6925`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:53.365456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-7b1bfcb0-1034-40a4-83a7-d90baa81eddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:53.475458 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:53.518640 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1362/2735] 10.222.0.78 () {58 vars in 1018 bytes} [Wed May 25 15:04:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:53.534694 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e448f9-c5fb-4d51-8282-ab60ec254f46 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:53.536500 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-55e448f9-c5fb-4d51-8282-ab60ec254f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:53.596118 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:53.676458 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e448f9-c5fb-4d51-8282-ab60ec254f46 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:53.682340 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae83cdf-790c-4afe-ad12-5efda07491d7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88cbd76f70d447c4ba92777b2c09821d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:53.683029 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae83cdf-790c-4afe-ad12-5efda07491d7 service glance] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:53.683318 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae83cdf-790c-4afe-ad12-5efda07491d7 service glance] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:53.683601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae83cdf-790c-4afe-ad12-5efda07491d7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:53.711906 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae83cdf-790c-4afe-ad12-5efda07491d7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:53.713617 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae83cdf-790c-4afe-ad12-5efda07491d7 service glance] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:53.750630 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1374/2736] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:53.856329 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c1334-bdac-4566-be79-a2d14cc64131 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:53.857123 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c1334-bdac-4566-be79-a2d14cc64131 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:53.857123 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c1334-bdac-4566-be79-a2d14cc64131 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:53.858542 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1363/2737] 10.222.0.78 () {54 vars in 921 bytes} [Wed May 25 15:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:04:53.864242 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafcc0a5-bc35-471b-97a6-bd2a37d50832 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:53.864579 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafcc0a5-bc35-471b-97a6-bd2a37d50832 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:53.864833 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafcc0a5-bc35-471b-97a6-bd2a37d50832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:53.896833 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-fafcc0a5-bc35-471b-97a6-bd2a37d50832 None None] MFA Rules not processed for user `70ad51a8ad68440da6618891e2e08cb8`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:53.899251 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-fafcc0a5-bc35-471b-97a6-bd2a37d50832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:53.926656 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1375/2738] 10.222.0.78 () {58 vars in 1018 bytes} [Wed May 25 15:04:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:04:53.978313 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c1334-bdac-4566-be79-a2d14cc64131 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:54.001824 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c1334-bdac-4566-be79-a2d14cc64131 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:54.028014 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe447f-5051-40da-a998-9e354f5f88e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:54.028682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe447f-5051-40da-a998-9e354f5f88e9 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:54.028964 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe447f-5051-40da-a998-9e354f5f88e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:54.029234 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe447f-5051-40da-a998-9e354f5f88e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:54.048360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe447f-5051-40da-a998-9e354f5f88e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:54.049477 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe447f-5051-40da-a998-9e354f5f88e9 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:54.067451 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1364/2739] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:54.105239 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafcc0a5-bc35-471b-97a6-bd2a37d50832 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:54.106632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-fafcc0a5-bc35-471b-97a6-bd2a37d50832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:54.228453 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafcc0a5-bc35-471b-97a6-bd2a37d50832 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:54.233847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed81c10-20c7-4ea7-b6e5-992d6f11d50b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:54.234797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed81c10-20c7-4ea7-b6e5-992d6f11d50b service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:54.234797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed81c10-20c7-4ea7-b6e5-992d6f11d50b service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:54.235202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed81c10-20c7-4ea7-b6e5-992d6f11d50b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:54.239580 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8ed81c10-20c7-4ea7-b6e5-992d6f11d50b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:54.360086 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1550565-d559-4584-be3b-10f916d097e8 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:54.361065 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1550565-d559-4584-be3b-10f916d097e8 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:54.361559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1550565-d559-4584-be3b-10f916d097e8 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:54.365530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1365/2740] 10.222.0.78 () {54 vars in 921 bytes} [Wed May 25 15:04:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:04:54.373101 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203e9e6-49ed-411d-9745-54ed3a2335f2 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:54.375153 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203e9e6-49ed-411d-9745-54ed3a2335f2 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:54.376426 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203e9e6-49ed-411d-9745-54ed3a2335f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:54.393105 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed81c10-20c7-4ea7-b6e5-992d6f11d50b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:54.394260 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed81c10-20c7-4ea7-b6e5-992d6f11d50b service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:54.418090 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1376/2741] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:54.440099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-b203e9e6-49ed-411d-9745-54ed3a2335f2 None None] MFA Rules not processed for user `e34e1d7ffbac49b0bde45829af0c6925`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:54.443123 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-b203e9e6-49ed-411d-9745-54ed3a2335f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:54.454940 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1366/2742] 10.222.0.78 () {58 vars in 1018 bytes} [Wed May 25 15:04:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:04:54.485237 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed81c10-20c7-4ea7-b6e5-992d6f11d50b service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:54.499882 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed81c10-20c7-4ea7-b6e5-992d6f11d50b service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:54.515174 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e212e-4ae0-4720-94b9-1a337ad904e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:54.516754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e212e-4ae0-4720-94b9-1a337ad904e1 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:54.517415 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e212e-4ae0-4720-94b9-1a337ad904e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:54.518000 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e212e-4ae0-4720-94b9-1a337ad904e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:54.549973 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e212e-4ae0-4720-94b9-1a337ad904e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:54.551992 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e212e-4ae0-4720-94b9-1a337ad904e1 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:54.573610 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1377/2743] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:04:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:55.470659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2e3b6-7b04-4f9f-beb3-c45cd6118331 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:55.471177 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2e3b6-7b04-4f9f-beb3-c45cd6118331 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:55.471303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2e3b6-7b04-4f9f-beb3-c45cd6118331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:55.507446 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-72d2e3b6-7b04-4f9f-beb3-c45cd6118331 None None] MFA Rules not processed for user `9c1bf697f0fd4e2cbc8d52cec772e446`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:55.508798 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-72d2e3b6-7b04-4f9f-beb3-c45cd6118331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:55.608002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1367/2744] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:55.612813 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e212e-4ae0-4720-94b9-1a337ad904e1 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:55.614486 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-de7e212e-4ae0-4720-94b9-1a337ad904e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:55.630325 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e212e-4ae0-4720-94b9-1a337ad904e1 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:55.636438 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ef38c-9f77-4025-afd7-493e3bdf3741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:55.637673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ef38c-9f77-4025-afd7-493e3bdf3741 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:55.638238 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ef38c-9f77-4025-afd7-493e3bdf3741 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:55.638737 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ef38c-9f77-4025-afd7-493e3bdf3741 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:55.640279 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ef38c-9f77-4025-afd7-493e3bdf3741 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:55.644331 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ef38c-9f77-4025-afd7-493e3bdf3741 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:55.677330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1378/2745] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:04:55 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:55.682000 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2e3b6-7b04-4f9f-beb3-c45cd6118331 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:55.697027 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2e3b6-7b04-4f9f-beb3-c45cd6118331 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:55.791340 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705aec5-8e20-4851-bfac-cacfbdd467b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:55.792408 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705aec5-8e20-4851-bfac-cacfbdd467b8 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:55.792852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705aec5-8e20-4851-bfac-cacfbdd467b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:55.793442 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705aec5-8e20-4851-bfac-cacfbdd467b8 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:55.794333 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5705aec5-8e20-4851-bfac-cacfbdd467b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:55.797206 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5705aec5-8e20-4851-bfac-cacfbdd467b8 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:55.824535 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1368/2746] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:04:55 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:55.828377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ef38c-9f77-4025-afd7-493e3bdf3741 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:55.841721 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ef38c-9f77-4025-afd7-493e3bdf3741 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:55.847466 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e44824-b896-4f58-9243-96646162d230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:55.848427 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e44824-b896-4f58-9243-96646162d230 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:55.848716 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e44824-b896-4f58-9243-96646162d230 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:55.849248 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e44824-b896-4f58-9243-96646162d230 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:55.849792 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e44824-b896-4f58-9243-96646162d230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:55.851512 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e44824-b896-4f58-9243-96646162d230 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:55.865840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1379/2747] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:55.870724 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705aec5-8e20-4851-bfac-cacfbdd467b8 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:55.884765 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705aec5-8e20-4851-bfac-cacfbdd467b8 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:55.892043 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f72e77-23a0-444d-a281-36c624ddc6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:55.893671 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f72e77-23a0-444d-a281-36c624ddc6f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:55.899831 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f72e77-23a0-444d-a281-36c624ddc6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:55.901660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f72e77-23a0-444d-a281-36c624ddc6f2 admin admin] PATH_INFO: `/v3/projects/11e7a4fad2d24584b2af7b50a4677ff4/users/e5081ba867b541d9ac2638a1738eb8b9/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:55.929698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f72e77-23a0-444d-a281-36c624ddc6f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11e7a4fad2d24584b2af7b50a4677ff4, user_id=e5081ba867b541d9ac2638a1738eb8b9, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:55.934617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f72e77-23a0-444d-a281-36c624ddc6f2 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:55.956104 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1369/2748] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:55 2022] PUT /identity/v3/projects/11e7a4fad2d24584b2af7b50a4677ff4/users/e5081ba867b541d9ac2638a1738eb8b9/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:55.957439 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e44824-b896-4f58-9243-96646162d230 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:55.969139 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e44824-b896-4f58-9243-96646162d230 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:56.042931 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:56.113636 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc55a9-5554-4c06-8dba-a0153ef399a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:56.114377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc55a9-5554-4c06-8dba-a0153ef399a9 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:56.114666 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc55a9-5554-4c06-8dba-a0153ef399a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:56.114948 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc55a9-5554-4c06-8dba-a0153ef399a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:56.115747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc55a9-5554-4c06-8dba-a0153ef399a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:56.117358 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc55a9-5554-4c06-8dba-a0153ef399a9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:56.145578 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1380/2749] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:56.145877 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f72e77-23a0-444d-a281-36c624ddc6f2 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:56.160184 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f72e77-23a0-444d-a281-36c624ddc6f2 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:56.174481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6332a1-07e7-4f4c-a6a6-329be51f14b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:56.175531 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6332a1-07e7-4f4c-a6a6-329be51f14b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:56.176327 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6332a1-07e7-4f4c-a6a6-329be51f14b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:56.176871 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6332a1-07e7-4f4c-a6a6-329be51f14b3 admin admin] PATH_INFO: `/v3/projects/11e7a4fad2d24584b2af7b50a4677ff4/users/e5081ba867b541d9ac2638a1738eb8b9/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:56.179353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6332a1-07e7-4f4c-a6a6-329be51f14b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11e7a4fad2d24584b2af7b50a4677ff4, user_id=e5081ba867b541d9ac2638a1738eb8b9, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:56.182460 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6332a1-07e7-4f4c-a6a6-329be51f14b3 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:56.201410 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1370/2750] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:56 2022] PUT /identity/v3/projects/11e7a4fad2d24584b2af7b50a4677ff4/users/e5081ba867b541d9ac2638a1738eb8b9/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:04:59.213072 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb261d-cee4-4d14-a798-a0c281f1c7ee None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:59.213619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb261d-cee4-4d14-a798-a0c281f1c7ee None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:59.213619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb261d-cee4-4d14-a798-a0c281f1c7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:59.265131 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-9feb261d-cee4-4d14-a798-a0c281f1c7ee None None] MFA Rules not processed for user `f892dfa98adb40198ce1296134d1263c`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:04:59.266989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9feb261d-cee4-4d14-a798-a0c281f1c7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:59.418104 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1381/2751] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:04:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:04:59.422843 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6332a1-07e7-4f4c-a6a6-329be51f14b3 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:59.448043 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:59.531557 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6332a1-07e7-4f4c-a6a6-329be51f14b3 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:59.537548 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c6338-a7ac-49b3-9c6d-876da541edf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:59.538421 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c6338-a7ac-49b3-9c6d-876da541edf8 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:59.538771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c6338-a7ac-49b3-9c6d-876da541edf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:59.540161 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c6338-a7ac-49b3-9c6d-876da541edf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:59.541012 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c6338-a7ac-49b3-9c6d-876da541edf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:59.543648 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c6338-a7ac-49b3-9c6d-876da541edf8 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:59.589108 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1371/2752] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:04:59 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:59.592898 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb261d-cee4-4d14-a798-a0c281f1c7ee None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:59.612406 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb261d-cee4-4d14-a798-a0c281f1c7ee None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:59.744240 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaafff13-a7a6-4397-8c8e-59d677636dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:59.745398 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafff13-a7a6-4397-8c8e-59d677636dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:59.746482 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafff13-a7a6-4397-8c8e-59d677636dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:59.747011 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafff13-a7a6-4397-8c8e-59d677636dd7 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:59.747898 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaafff13-a7a6-4397-8c8e-59d677636dd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:59.751009 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaafff13-a7a6-4397-8c8e-59d677636dd7 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:59.784543 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1382/2753] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:04:59 2022] POST /identity/v3/users => generated 299 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:04:59.788385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c6338-a7ac-49b3-9c6d-876da541edf8 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:59.804176 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c6338-a7ac-49b3-9c6d-876da541edf8 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:59.829956 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836e2d2-2b81-4879-9b91-bdd49bd0746d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:59.830692 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836e2d2-2b81-4879-9b91-bdd49bd0746d admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:59.830973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836e2d2-2b81-4879-9b91-bdd49bd0746d admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:59.831247 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836e2d2-2b81-4879-9b91-bdd49bd0746d admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:59.831996 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836e2d2-2b81-4879-9b91-bdd49bd0746d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:59.834064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836e2d2-2b81-4879-9b91-bdd49bd0746d admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:59.852843 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1372/2754] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:04:59.854215 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaafff13-a7a6-4397-8c8e-59d677636dd7 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:59.872263 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaafff13-a7a6-4397-8c8e-59d677636dd7 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:59.877392 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69677a6-2ee8-4a02-ad97-7971010c1aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:59.878393 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69677a6-2ee8-4a02-ad97-7971010c1aee admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:59.878708 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69677a6-2ee8-4a02-ad97-7971010c1aee admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:59.880383 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69677a6-2ee8-4a02-ad97-7971010c1aee admin admin] PATH_INFO: `/v3/projects/34062b9020a74975b3a0eb31030e4fa6/users/7dac312060bd47d9869f3502ced1dda0/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:59.907594 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69677a6-2ee8-4a02-ad97-7971010c1aee admin admin] RBAC: Authorizing `identity:create_grant(project_id=34062b9020a74975b3a0eb31030e4fa6, user_id=7dac312060bd47d9869f3502ced1dda0, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:59.910105 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69677a6-2ee8-4a02-ad97-7971010c1aee admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:59.935326 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1383/2755] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:04:59 2022] PUT /identity/v3/projects/34062b9020a74975b3a0eb31030e4fa6/users/7dac312060bd47d9869f3502ced1dda0/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:04:59.940202 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836e2d2-2b81-4879-9b91-bdd49bd0746d admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:59.952460 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836e2d2-2b81-4879-9b91-bdd49bd0746d admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:00.086833 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27567d0-cddb-44ff-ae85-59d4f80bde07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:00.087526 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27567d0-cddb-44ff-ae85-59d4f80bde07 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:00.087804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27567d0-cddb-44ff-ae85-59d4f80bde07 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:00.088229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27567d0-cddb-44ff-ae85-59d4f80bde07 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:00.088974 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27567d0-cddb-44ff-ae85-59d4f80bde07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:00.090640 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27567d0-cddb-44ff-ae85-59d4f80bde07 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:00.120232 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69677a6-2ee8-4a02-ad97-7971010c1aee admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:00.122240 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1373/2756] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:05:00.135437 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69677a6-2ee8-4a02-ad97-7971010c1aee admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:00.147618 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea03acd-05ad-45a8-bef8-293129dc842b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:00.148692 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea03acd-05ad-45a8-bef8-293129dc842b admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:00.148973 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea03acd-05ad-45a8-bef8-293129dc842b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:00.149246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea03acd-05ad-45a8-bef8-293129dc842b admin admin] PATH_INFO: `/v3/projects/34062b9020a74975b3a0eb31030e4fa6/users/7dac312060bd47d9869f3502ced1dda0/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:00.151294 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea03acd-05ad-45a8-bef8-293129dc842b admin admin] RBAC: Authorizing `identity:create_grant(project_id=34062b9020a74975b3a0eb31030e4fa6, user_id=7dac312060bd47d9869f3502ced1dda0, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:00.157295 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea03acd-05ad-45a8-bef8-293129dc842b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:00.190405 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1384/2757] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:05:00 2022] PUT /identity/v3/projects/34062b9020a74975b3a0eb31030e4fa6/users/7dac312060bd47d9869f3502ced1dda0/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:05:01.958969 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aebe39-384f-458b-89b1-7c6474f9b29c None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:01.959469 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aebe39-384f-458b-89b1-7c6474f9b29c None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:01.959584 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aebe39-384f-458b-89b1-7c6474f9b29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:01.996892 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-a2aebe39-384f-458b-89b1-7c6474f9b29c None None] MFA Rules not processed for user `daec78050ee547699910603eff8cb7ca`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:01.998159 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-a2aebe39-384f-458b-89b1-7c6474f9b29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:02.108099 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a8513-0bd0-407c-b747-6d35b8d4063d None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:02.109304 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a8513-0bd0-407c-b747-6d35b8d4063d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:02.110069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a8513-0bd0-407c-b747-6d35b8d4063d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:02.133629 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:02.140879 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1374/2758] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:05:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:02.162508 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-403a8513-0bd0-407c-b747-6d35b8d4063d None None] MFA Rules not processed for user `b2935579c8554bed8b1e211c47548ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:02.166717 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-403a8513-0bd0-407c-b747-6d35b8d4063d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:02.308821 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1385/2759] 10.222.0.78 () {58 vars in 1009 bytes} [Wed May 25 15:05:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:05:02.324251 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aebe39-384f-458b-89b1-7c6474f9b29c None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:02.325919 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-a2aebe39-384f-458b-89b1-7c6474f9b29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:02.365113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:02.444726 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aebe39-384f-458b-89b1-7c6474f9b29c None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:02.452115 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bbaaa9-e78f-467e-81fb-d969ce158b98 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe4a2d106d84fc9ac65c046ba675353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:02.453537 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbaaa9-e78f-467e-81fb-d969ce158b98 service swift] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:02.454181 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbaaa9-e78f-467e-81fb-d969ce158b98 service swift] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:02.454682 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbaaa9-e78f-467e-81fb-d969ce158b98 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:02.466747 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bbaaa9-e78f-467e-81fb-d969ce158b98 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:02.468705 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bbaaa9-e78f-467e-81fb-d969ce158b98 service swift] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:02.479423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1375/2760] 10.222.0.78 () {60 vars in 1384 bytes} [Wed May 25 15:05:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 15:05:03.145729 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a8513-0bd0-407c-b747-6d35b8d4063d None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:03.325915 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a8513-0bd0-407c-b747-6d35b8d4063d None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:03.337671 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74e7a6-e88f-492a-9e1d-ef004b863fca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:03.338366 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74e7a6-e88f-492a-9e1d-ef004b863fca service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:03.338756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74e7a6-e88f-492a-9e1d-ef004b863fca service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:03.339153 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74e7a6-e88f-492a-9e1d-ef004b863fca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:03.346096 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2d74e7a6-e88f-492a-9e1d-ef004b863fca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:03.399736 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74e7a6-e88f-492a-9e1d-ef004b863fca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:03.404352 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74e7a6-e88f-492a-9e1d-ef004b863fca service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:03.449344 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1386/2761] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:05:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:05:08.013115 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5466f-dfbf-4292-bade-182fb8947c9f None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:08.013754 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5466f-dfbf-4292-bade-182fb8947c9f None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:08.014613 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5466f-dfbf-4292-bade-182fb8947c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:08.084900 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-12a5466f-dfbf-4292-bade-182fb8947c9f None None] MFA Rules not processed for user `e5081ba867b541d9ac2638a1738eb8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:08.086307 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-12a5466f-dfbf-4292-bade-182fb8947c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:08.281910 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1376/2762] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:05:08 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:08.296170 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74e7a6-e88f-492a-9e1d-ef004b863fca service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:08.297866 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2d74e7a6-e88f-492a-9e1d-ef004b863fca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:08.491180 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74e7a6-e88f-492a-9e1d-ef004b863fca service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:08.500578 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7253b6d-27bc-4378-b82f-60dedf5dc94d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:08.501768 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7253b6d-27bc-4378-b82f-60dedf5dc94d service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:08.502278 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7253b6d-27bc-4378-b82f-60dedf5dc94d service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:08.502609 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7253b6d-27bc-4378-b82f-60dedf5dc94d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:08.609035 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:08.622840 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7253b6d-27bc-4378-b82f-60dedf5dc94d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:08.626708 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7253b6d-27bc-4378-b82f-60dedf5dc94d service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:08.642110 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1387/2763] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:05:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:05:11.595760 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72aee4-275a-436f-a6b8-ccb623ceb7c2 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:11.596383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72aee4-275a-436f-a6b8-ccb623ceb7c2 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:11.596472 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72aee4-275a-436f-a6b8-ccb623ceb7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:11.660829 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-1b72aee4-275a-436f-a6b8-ccb623ceb7c2 None None] MFA Rules not processed for user `7dac312060bd47d9869f3502ced1dda0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:11.662125 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-1b72aee4-275a-436f-a6b8-ccb623ceb7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:11.834334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1377/2764] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:05:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:11.845641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7253b6d-27bc-4378-b82f-60dedf5dc94d service nova] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:12.030689 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7253b6d-27bc-4378-b82f-60dedf5dc94d service nova] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:12.042241 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e947b-58df-4579-8f14-31acf9613ccb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88cbd76f70d447c4ba92777b2c09821d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:12.044328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e947b-58df-4579-8f14-31acf9613ccb service glance] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:12.044622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e947b-58df-4579-8f14-31acf9613ccb service glance] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:12.045185 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e947b-58df-4579-8f14-31acf9613ccb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:12.247424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e947b-58df-4579-8f14-31acf9613ccb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:12.256545 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e947b-58df-4579-8f14-31acf9613ccb service glance] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:12.319112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1388/2765] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:05:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:05:12.620890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d76d3f-f9d1-4b97-ae1b-6fe162180004 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:12.621468 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d76d3f-f9d1-4b97-ae1b-6fe162180004 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:12.621766 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d76d3f-f9d1-4b97-ae1b-6fe162180004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:12.660398 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-a3d76d3f-f9d1-4b97-ae1b-6fe162180004 None None] MFA Rules not processed for user `b2935579c8554bed8b1e211c47548ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:12.662072 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-a3d76d3f-f9d1-4b97-ae1b-6fe162180004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:12.699527 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1378/2766] 10.222.0.78 () {58 vars in 1009 bytes} [Wed May 25 15:05:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:12.724772 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e947b-58df-4579-8f14-31acf9613ccb service glance] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:12.779342 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e947b-58df-4579-8f14-31acf9613ccb service glance] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:12.804136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca05e9-56e4-46a1-8779-9d6051f2fa27 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe4a2d106d84fc9ac65c046ba675353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:12.805185 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca05e9-56e4-46a1-8779-9d6051f2fa27 service swift] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:12.805634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca05e9-56e4-46a1-8779-9d6051f2fa27 service swift] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:12.806420 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca05e9-56e4-46a1-8779-9d6051f2fa27 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:12.830474 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca05e9-56e4-46a1-8779-9d6051f2fa27 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:12.832640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca05e9-56e4-46a1-8779-9d6051f2fa27 service swift] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:12.855576 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1389/2767] 10.222.0.78 () {60 vars in 1384 bytes} [Wed May 25 15:05:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 15:05:13.201883 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d1db8-983f-49f5-8f64-25612954f7d1 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:13.202224 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d1db8-983f-49f5-8f64-25612954f7d1 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:13.202572 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d1db8-983f-49f5-8f64-25612954f7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:13.246509 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-3f2d1db8-983f-49f5-8f64-25612954f7d1 None None] MFA Rules not processed for user `9c1bf697f0fd4e2cbc8d52cec772e446`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:13.252211 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-3f2d1db8-983f-49f5-8f64-25612954f7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:13.278594 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1379/2768] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:05:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:22.849726 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=58 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:23.227643 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=95 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:25.477129 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe7349-8c03-4cb5-870a-847867c10bc3 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:25.477738 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe7349-8c03-4cb5-870a-847867c10bc3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:25.478082 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe7349-8c03-4cb5-870a-847867c10bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:25.497965 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-f2fe7349-8c03-4cb5-870a-847867c10bc3 None None] MFA Rules not processed for user `f892dfa98adb40198ce1296134d1263c`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:25.499129 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f2fe7349-8c03-4cb5-870a-847867c10bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:25.506566 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1390/2769] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:05:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:28.692048 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14ccff-935a-4be3-96ed-7c52c85e4ffd None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:28.692581 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14ccff-935a-4be3-96ed-7c52c85e4ffd None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:28.692835 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14ccff-935a-4be3-96ed-7c52c85e4ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:28.721542 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-ed14ccff-935a-4be3-96ed-7c52c85e4ffd None None] MFA Rules not processed for user `b2935579c8554bed8b1e211c47548ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:28.722818 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-ed14ccff-935a-4be3-96ed-7c52c85e4ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:28.730619 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe7349-8c03-4cb5-870a-847867c10bc3 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:28.732607 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f2fe7349-8c03-4cb5-870a-847867c10bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:28.736246 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1380/2770] 10.222.0.78 () {58 vars in 1009 bytes} [Wed May 25 15:05:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:28.745729 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe7349-8c03-4cb5-870a-847867c10bc3 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:28.750749 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945d4f1-3de8-4a54-894d-378736f8bbbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:28.751003 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14ccff-935a-4be3-96ed-7c52c85e4ffd None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:28.751943 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945d4f1-3de8-4a54-894d-378736f8bbbb service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:28.753298 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945d4f1-3de8-4a54-894d-378736f8bbbb service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:28.753999 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945d4f1-3de8-4a54-894d-378736f8bbbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:28.756527 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3945d4f1-3de8-4a54-894d-378736f8bbbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:28.763977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14ccff-935a-4be3-96ed-7c52c85e4ffd None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:28.770598 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f51ef1-76b8-49d1-a646-481af1b6efb4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe4a2d106d84fc9ac65c046ba675353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:28.771310 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f51ef1-76b8-49d1-a646-481af1b6efb4 service swift] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:28.771916 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f51ef1-76b8-49d1-a646-481af1b6efb4 service swift] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:28.771916 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f51ef1-76b8-49d1-a646-481af1b6efb4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:28.788379 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f51ef1-76b8-49d1-a646-481af1b6efb4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:28.790068 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f51ef1-76b8-49d1-a646-481af1b6efb4 service swift] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:28.799799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1381/2771] 10.222.0.78 () {60 vars in 1384 bytes} [Wed May 25 15:05:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 15:05:28.855791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3945d4f1-3de8-4a54-894d-378736f8bbbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:28.857974 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3945d4f1-3de8-4a54-894d-378736f8bbbb service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:28.890461 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1391/2772] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:05:28 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:05:38.796564 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=9 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:38.872263 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=33 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:41.570383 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183cc4a-35aa-44b6-9f39-2aff377d6e13 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:41.570919 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183cc4a-35aa-44b6-9f39-2aff377d6e13 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:41.570919 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183cc4a-35aa-44b6-9f39-2aff377d6e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:41.589854 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-6183cc4a-35aa-44b6-9f39-2aff377d6e13 None None] MFA Rules not processed for user `b2935579c8554bed8b1e211c47548ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:41.591200 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-6183cc4a-35aa-44b6-9f39-2aff377d6e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:41.599334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1382/2773] 10.222.0.78 () {58 vars in 967 bytes} [Wed May 25 15:05:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:41.626601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945d4f1-3de8-4a54-894d-378736f8bbbb service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:41.644557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945d4f1-3de8-4a54-894d-378736f8bbbb service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:41.649865 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4da69d-388d-449d-9d77-084b05d7fa77 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe4a2d106d84fc9ac65c046ba675353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:41.650570 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4da69d-388d-449d-9d77-084b05d7fa77 service swift] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:41.650855 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4da69d-388d-449d-9d77-084b05d7fa77 service swift] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:41.651131 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4da69d-388d-449d-9d77-084b05d7fa77 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:41.660901 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4da69d-388d-449d-9d77-084b05d7fa77 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:41.662549 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4da69d-388d-449d-9d77-084b05d7fa77 service swift] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:41.671973 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1392/2774] 10.222.0.78 () {60 vars in 1384 bytes} [Wed May 25 15:05:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 15:05:51.584397 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:51.668021 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=6 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:58.691768 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183cc4a-35aa-44b6-9f39-2aff377d6e13 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:58.701281 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183cc4a-35aa-44b6-9f39-2aff377d6e13 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:58.704819 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75587fe-a350-4b21-a3ad-12a84bad934f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:58.705590 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75587fe-a350-4b21-a3ad-12a84bad934f service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:58.705871 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75587fe-a350-4b21-a3ad-12a84bad934f service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:58.706185 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75587fe-a350-4b21-a3ad-12a84bad934f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:58.715258 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75587fe-a350-4b21-a3ad-12a84bad934f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:58.716685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75587fe-a350-4b21-a3ad-12a84bad934f service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:58.726336 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1383/2775] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:05:58 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:01.818471 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4da69d-388d-449d-9d77-084b05d7fa77 service swift] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:01.828067 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4da69d-388d-449d-9d77-084b05d7fa77 service swift] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:01.831652 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c37f1-2dd3-4f37-bf19-5c3f51788e1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:01.832294 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c37f1-2dd3-4f37-bf19-5c3f51788e1f service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:01.832573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c37f1-2dd3-4f37-bf19-5c3f51788e1f service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:01.832841 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c37f1-2dd3-4f37-bf19-5c3f51788e1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:01.842547 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c37f1-2dd3-4f37-bf19-5c3f51788e1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:01.843995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c37f1-2dd3-4f37-bf19-5c3f51788e1f service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:01.854422 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1393/2776] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:06:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:05.175404 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75587fe-a350-4b21-a3ad-12a84bad934f service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:05.186525 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75587fe-a350-4b21-a3ad-12a84bad934f service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:05.191264 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e66776-9c32-43ab-916a-31d62a90a394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:05.192077 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e66776-9c32-43ab-916a-31d62a90a394 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:05.192427 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e66776-9c32-43ab-916a-31d62a90a394 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:05.192772 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e66776-9c32-43ab-916a-31d62a90a394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:05.194968 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-83e66776-9c32-43ab-916a-31d62a90a394 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:05.211396 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e66776-9c32-43ab-916a-31d62a90a394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:05.213027 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e66776-9c32-43ab-916a-31d62a90a394 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:05.225956 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1384/2777] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:06:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:06:05.862222 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c37f1-2dd3-4f37-bf19-5c3f51788e1f service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:05.871755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c37f1-2dd3-4f37-bf19-5c3f51788e1f service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:05.875794 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc2b14-c8bf-404f-8353-0325698d960d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:05.876475 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc2b14-c8bf-404f-8353-0325698d960d service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:05.877293 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc2b14-c8bf-404f-8353-0325698d960d service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:05.877571 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc2b14-c8bf-404f-8353-0325698d960d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:05.879682 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-93fc2b14-c8bf-404f-8353-0325698d960d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:05.896881 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc2b14-c8bf-404f-8353-0325698d960d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:05.898492 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc2b14-c8bf-404f-8353-0325698d960d service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:05.913574 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1394/2778] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:06:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:06:15.221063 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=12 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:15.906466 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=12 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:40.994384 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e66776-9c32-43ab-916a-31d62a90a394 service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:41.007337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e66776-9c32-43ab-916a-31d62a90a394 service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:41.013572 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce4417-57b2-4b27-b537-c19a84856044 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:41.014271 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce4417-57b2-4b27-b537-c19a84856044 service nova] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:41.014560 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce4417-57b2-4b27-b537-c19a84856044 service nova] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:41.015036 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce4417-57b2-4b27-b537-c19a84856044 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:41.016872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-72ce4417-57b2-4b27-b537-c19a84856044 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:41.030656 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ce4417-57b2-4b27-b537-c19a84856044 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:41.032262 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ce4417-57b2-4b27-b537-c19a84856044 service nova] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:41.043910 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1385/2779] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:06:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:41.690166 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc2b14-c8bf-404f-8353-0325698d960d service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:41.700262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc2b14-c8bf-404f-8353-0325698d960d service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:41.704091 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f5a51-fe85-49eb-b388-f98f5aeff1cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:41.705049 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f5a51-fe85-49eb-b388-f98f5aeff1cf service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:41.705536 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f5a51-fe85-49eb-b388-f98f5aeff1cf service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:41.705807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f5a51-fe85-49eb-b388-f98f5aeff1cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:41.708290 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ab0f5a51-fe85-49eb-b388-f98f5aeff1cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:41.719920 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f5a51-fe85-49eb-b388-f98f5aeff1cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:41.721688 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f5a51-fe85-49eb-b388-f98f5aeff1cf service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:41.730989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1395/2780] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:06:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:43.870267 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04e888-1cb1-4db7-86cc-c15538a73a68 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:43.870821 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04e888-1cb1-4db7-86cc-c15538a73a68 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:43.870821 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04e888-1cb1-4db7-86cc-c15538a73a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:43.889360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-4d04e888-1cb1-4db7-86cc-c15538a73a68 None None] MFA Rules not processed for user `34b1591b0eb944ed98c42d35a03c1ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:43.890600 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-4d04e888-1cb1-4db7-86cc-c15538a73a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:43.947655 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1386/2781] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:06:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:44.574834 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f5a51-fe85-49eb-b388-f98f5aeff1cf service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:44.576233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ab0f5a51-fe85-49eb-b388-f98f5aeff1cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:44.651991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f5a51-fe85-49eb-b388-f98f5aeff1cf service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:44.657119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed1613-7f3d-43d4-bb5e-5359f0aff8ed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63789c0a113447ff90933d447cf5bac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:44.657803 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed1613-7f3d-43d4-bb5e-5359f0aff8ed service placement] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:44.658120 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed1613-7f3d-43d4-bb5e-5359f0aff8ed service placement] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:44.658401 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed1613-7f3d-43d4-bb5e-5359f0aff8ed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:44.660208 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e7ed1613-7f3d-43d4-bb5e-5359f0aff8ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:44.671798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed1613-7f3d-43d4-bb5e-5359f0aff8ed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:44.673407 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed1613-7f3d-43d4-bb5e-5359f0aff8ed service placement] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:44.683411 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1396/2782] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:06:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:53.940637 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=31 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:54.679782 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=38 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:07:40.522989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04e888-1cb1-4db7-86cc-c15538a73a68 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:40.618623 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04e888-1cb1-4db7-86cc-c15538a73a68 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:40.626533 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28580615-a07b-40b1-9d05-a9dbb208e170 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:40.627582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28580615-a07b-40b1-9d05-a9dbb208e170 service nova] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:40.628099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28580615-a07b-40b1-9d05-a9dbb208e170 service nova] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:40.628752 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28580615-a07b-40b1-9d05-a9dbb208e170 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:40.631670 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-28580615-a07b-40b1-9d05-a9dbb208e170 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:40.708324 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28580615-a07b-40b1-9d05-a9dbb208e170 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:40.712480 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28580615-a07b-40b1-9d05-a9dbb208e170 service nova] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:40.738422 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1387/2783] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:07:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:07:42.095592 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed1613-7f3d-43d4-bb5e-5359f0aff8ed service placement] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:42.184234 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed1613-7f3d-43d4-bb5e-5359f0aff8ed service placement] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:42.190377 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409d5bc-39a2-4268-88eb-72919279e70c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:42.191086 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409d5bc-39a2-4268-88eb-72919279e70c service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:42.191454 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409d5bc-39a2-4268-88eb-72919279e70c service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:42.191972 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409d5bc-39a2-4268-88eb-72919279e70c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:42.194369 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9409d5bc-39a2-4268-88eb-72919279e70c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:42.375015 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9409d5bc-39a2-4268-88eb-72919279e70c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:42.376719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9409d5bc-39a2-4268-88eb-72919279e70c service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:42.404483 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1397/2784] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:07:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:07:50.102681 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b8cd5-dbff-4f13-907e-06e1a0642467 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:50.103615 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b8cd5-dbff-4f13-907e-06e1a0642467 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:50.104093 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b8cd5-dbff-4f13-907e-06e1a0642467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:50.133436 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-ca0b8cd5-dbff-4f13-907e-06e1a0642467 None None] MFA Rules not processed for user `b2935579c8554bed8b1e211c47548ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:07:50.134988 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-ca0b8cd5-dbff-4f13-907e-06e1a0642467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:50.153916 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1388/2785] 10.222.0.78 () {58 vars in 967 bytes} [Wed May 25 15:07:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:07:50.170238 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409d5bc-39a2-4268-88eb-72919279e70c service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:50.184952 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409d5bc-39a2-4268-88eb-72919279e70c service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:50.191303 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e4c45-1989-4d2b-a220-78edf05fac9a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63789c0a113447ff90933d447cf5bac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:50.192436 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e4c45-1989-4d2b-a220-78edf05fac9a service placement] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:50.192830 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e4c45-1989-4d2b-a220-78edf05fac9a service placement] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:50.193188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e4c45-1989-4d2b-a220-78edf05fac9a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:50.195632 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-111e4c45-1989-4d2b-a220-78edf05fac9a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:50.218755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111e4c45-1989-4d2b-a220-78edf05fac9a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:50.220344 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111e4c45-1989-4d2b-a220-78edf05fac9a service placement] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:50.233320 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b8cd5-dbff-4f13-907e-06e1a0642467 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:50.236062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1398/2786] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:07:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:07:50.238171 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-ca0b8cd5-dbff-4f13-907e-06e1a0642467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:50.240156 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e4c45-1989-4d2b-a220-78edf05fac9a service placement] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:50.259904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b8cd5-dbff-4f13-907e-06e1a0642467 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:50.263165 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e4c45-1989-4d2b-a220-78edf05fac9a service placement] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:50.269465 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a2a1d-bd1e-4258-b64d-b150ff96df86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe4a2d106d84fc9ac65c046ba675353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:50.270870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a2a1d-bd1e-4258-b64d-b150ff96df86 service swift] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:50.271353 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a2a1d-bd1e-4258-b64d-b150ff96df86 service swift] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:50.271807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a2a1d-bd1e-4258-b64d-b150ff96df86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:50.271949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d411d40-08ff-482e-9ac3-dcb2ca156265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:50.273002 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d411d40-08ff-482e-9ac3-dcb2ca156265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:50.273413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d411d40-08ff-482e-9ac3-dcb2ca156265 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:50.273669 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d411d40-08ff-482e-9ac3-dcb2ca156265 admin admin] PATH_INFO: `/v3/users/daec78050ee547699910603eff8cb7ca` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:50.275489 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d411d40-08ff-482e-9ac3-dcb2ca156265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=daec78050ee547699910603eff8cb7ca)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:50.277659 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d411d40-08ff-482e-9ac3-dcb2ca156265 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:50.283358 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a2a1d-bd1e-4258-b64d-b150ff96df86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:50.285274 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a2a1d-bd1e-4258-b64d-b150ff96df86 service swift] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:50.294566 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1399/2787] 10.222.0.78 () {60 vars in 1384 bytes} [Wed May 25 15:07:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 15:07:50.338547 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6d411d40-08ff-482e-9ac3-dcb2ca156265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daec78050ee547699910603eff8cb7ca'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:50.353604 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6d411d40-08ff-482e-9ac3-dcb2ca156265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daec78050ee547699910603eff8cb7ca'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:50.363799 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6d411d40-08ff-482e-9ac3-dcb2ca156265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daec78050ee547699910603eff8cb7ca'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:50.364568 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6d411d40-08ff-482e-9ac3-dcb2ca156265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daec78050ee547699910603eff8cb7ca'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:50.365236 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6d411d40-08ff-482e-9ac3-dcb2ca156265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daec78050ee547699910603eff8cb7ca'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:50.379594 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1389/2788] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:07:50 2022] DELETE /identity/v3/users/daec78050ee547699910603eff8cb7ca => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:07:50.579261 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a2a1d-bd1e-4258-b64d-b150ff96df86 service swift] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:50.592242 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a2a1d-bd1e-4258-b64d-b150ff96df86 service swift] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:50.660136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb8711-5cf3-4a8c-a26a-9f91da54e976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:50.660930 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb8711-5cf3-4a8c-a26a-9f91da54e976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:50.661220 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb8711-5cf3-4a8c-a26a-9f91da54e976 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:50.661503 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb8711-5cf3-4a8c-a26a-9f91da54e976 admin admin] PATH_INFO: `/v3/projects/99281bdc8f6340a1be03774c255ae4ba` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:50.662823 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb8711-5cf3-4a8c-a26a-9f91da54e976 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99281bdc8f6340a1be03774c255ae4ba)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:50.664766 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb8711-5cf3-4a8c-a26a-9f91da54e976 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:50.685740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:07:50.725406 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-b9cb8711-5cf3-4a8c-a26a-9f91da54e976 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99281bdc8f6340a1be03774c255ae4ba', 'request_id': 'req-b9cb8711-5cf3-4a8c-a26a-9f91da54e976'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:50.731069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-b9cb8711-5cf3-4a8c-a26a-9f91da54e976 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99281bdc8f6340a1be03774c255ae4ba', 'request_id': 'req-b9cb8711-5cf3-4a8c-a26a-9f91da54e976'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:50.737719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-b9cb8711-5cf3-4a8c-a26a-9f91da54e976 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99281bdc8f6340a1be03774c255ae4ba', 'request_id': 'req-b9cb8711-5cf3-4a8c-a26a-9f91da54e976'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:50.753448 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-b9cb8711-5cf3-4a8c-a26a-9f91da54e976 admin admin] The token cache is being invalidate because project 99281bdc8f6340a1be03774c255ae4ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:07:50.753836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-b9cb8711-5cf3-4a8c-a26a-9f91da54e976 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:50.754449 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-b9cb8711-5cf3-4a8c-a26a-9f91da54e976 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:50.756282 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1400/2789] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:07:50 2022] DELETE /identity/v3/projects/99281bdc8f6340a1be03774c255ae4ba => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:07:50.761138 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d411d40-08ff-482e-9ac3-dcb2ca156265 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:50.763630 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-6d411d40-08ff-482e-9ac3-dcb2ca156265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:50.779890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d411d40-08ff-482e-9ac3-dcb2ca156265 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:50.846845 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:07:50.861259 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c233dbd-b0b4-4c12-a768-4795a3d3a0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:50.863053 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c233dbd-b0b4-4c12-a768-4795a3d3a0c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:50.863222 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c233dbd-b0b4-4c12-a768-4795a3d3a0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:50.863991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c233dbd-b0b4-4c12-a768-4795a3d3a0c3 admin admin] PATH_INFO: `/v3/users/3712ad1188fe405b9b25d3f1254d6746` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:50.866197 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c233dbd-b0b4-4c12-a768-4795a3d3a0c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3712ad1188fe405b9b25d3f1254d6746)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:50.869515 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c233dbd-b0b4-4c12-a768-4795a3d3a0c3 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:50.941078 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6c233dbd-b0b4-4c12-a768-4795a3d3a0c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3712ad1188fe405b9b25d3f1254d6746'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:50.962743 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6c233dbd-b0b4-4c12-a768-4795a3d3a0c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3712ad1188fe405b9b25d3f1254d6746'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:50.973571 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6c233dbd-b0b4-4c12-a768-4795a3d3a0c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3712ad1188fe405b9b25d3f1254d6746'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:50.974315 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6c233dbd-b0b4-4c12-a768-4795a3d3a0c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3712ad1188fe405b9b25d3f1254d6746'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:50.975017 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-6c233dbd-b0b4-4c12-a768-4795a3d3a0c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3712ad1188fe405b9b25d3f1254d6746'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:50.992944 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1390/2790] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:07:50 2022] DELETE /identity/v3/users/3712ad1188fe405b9b25d3f1254d6746 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:07:51.228138 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb8711-5cf3-4a8c-a26a-9f91da54e976 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:51.229715 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b9cb8711-5cf3-4a8c-a26a-9f91da54e976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:51.247389 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb8711-5cf3-4a8c-a26a-9f91da54e976 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:51.317808 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a74d7-0800-491a-b96e-c41afccaeeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:51.319105 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a74d7-0800-491a-b96e-c41afccaeeaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:51.319370 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a74d7-0800-491a-b96e-c41afccaeeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:51.319645 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a74d7-0800-491a-b96e-c41afccaeeaf admin admin] PATH_INFO: `/v3/projects/9ce506a9ffd44d5fbe080b9fb2f2b693` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:51.320929 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a74d7-0800-491a-b96e-c41afccaeeaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ce506a9ffd44d5fbe080b9fb2f2b693)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:51.323697 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a74d7-0800-491a-b96e-c41afccaeeaf admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:51.386356 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-188a74d7-0800-491a-b96e-c41afccaeeaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ce506a9ffd44d5fbe080b9fb2f2b693', 'request_id': 'req-188a74d7-0800-491a-b96e-c41afccaeeaf'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:51.395827 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-188a74d7-0800-491a-b96e-c41afccaeeaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ce506a9ffd44d5fbe080b9fb2f2b693', 'request_id': 'req-188a74d7-0800-491a-b96e-c41afccaeeaf'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:51.402783 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-188a74d7-0800-491a-b96e-c41afccaeeaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ce506a9ffd44d5fbe080b9fb2f2b693', 'request_id': 'req-188a74d7-0800-491a-b96e-c41afccaeeaf'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:51.420431 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-188a74d7-0800-491a-b96e-c41afccaeeaf admin admin] The token cache is being invalidate because project 9ce506a9ffd44d5fbe080b9fb2f2b693 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:07:51.420807 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-188a74d7-0800-491a-b96e-c41afccaeeaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:51.421663 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-188a74d7-0800-491a-b96e-c41afccaeeaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:51.423513 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1401/2791] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:07:51 2022] DELETE /identity/v3/projects/9ce506a9ffd44d5fbe080b9fb2f2b693 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:07:56.042536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c233dbd-b0b4-4c12-a768-4795a3d3a0c3 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:56.044134 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-6c233dbd-b0b4-4c12-a768-4795a3d3a0c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:56.179109 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c233dbd-b0b4-4c12-a768-4795a3d3a0c3 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:56.251816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:07:56.274042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea0ee0-6331-45df-bca7-d99a4ca45914 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:56.274735 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea0ee0-6331-45df-bca7-d99a4ca45914 service nova] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:56.275022 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea0ee0-6331-45df-bca7-d99a4ca45914 service nova] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:56.275318 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea0ee0-6331-45df-bca7-d99a4ca45914 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:56.277261 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-49ea0ee0-6331-45df-bca7-d99a4ca45914 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:56.350137 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea0ee0-6331-45df-bca7-d99a4ca45914 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:56.351786 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea0ee0-6331-45df-bca7-d99a4ca45914 service nova] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:56.364530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1391/2792] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:07:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:07:57.883829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a74d7-0800-491a-b96e-c41afccaeeaf admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:57.885427 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-188a74d7-0800-491a-b96e-c41afccaeeaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:57.927745 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:07:57.961287 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a74d7-0800-491a-b96e-c41afccaeeaf admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:57.966136 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a22cb3-ce43-45df-9f68-b3f7047893a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:57.967047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a22cb3-ce43-45df-9f68-b3f7047893a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:57.967408 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a22cb3-ce43-45df-9f68-b3f7047893a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:57.967706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a22cb3-ce43-45df-9f68-b3f7047893a8 admin admin] PATH_INFO: `/v3/users/f892dfa98adb40198ce1296134d1263c` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:57.968938 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a22cb3-ce43-45df-9f68-b3f7047893a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f892dfa98adb40198ce1296134d1263c)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:57.970976 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a22cb3-ce43-45df-9f68-b3f7047893a8 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:58.020428 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-95a22cb3-ce43-45df-9f68-b3f7047893a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f892dfa98adb40198ce1296134d1263c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:58.032777 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-95a22cb3-ce43-45df-9f68-b3f7047893a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f892dfa98adb40198ce1296134d1263c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:58.040328 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-95a22cb3-ce43-45df-9f68-b3f7047893a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f892dfa98adb40198ce1296134d1263c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:58.041001 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-95a22cb3-ce43-45df-9f68-b3f7047893a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f892dfa98adb40198ce1296134d1263c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:58.041618 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-95a22cb3-ce43-45df-9f68-b3f7047893a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f892dfa98adb40198ce1296134d1263c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:58.056776 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1402/2793] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:07:57 2022] DELETE /identity/v3/users/f892dfa98adb40198ce1296134d1263c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:07:58.259794 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea0ee0-6331-45df-bca7-d99a4ca45914 service nova] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:58.271242 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea0ee0-6331-45df-bca7-d99a4ca45914 service nova] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:58.330240 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ddf69-ec56-49ee-925a-421767895344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:58.331108 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ddf69-ec56-49ee-925a-421767895344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:58.331379 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ddf69-ec56-49ee-925a-421767895344 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:58.331648 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ddf69-ec56-49ee-925a-421767895344 admin admin] PATH_INFO: `/v3/projects/5e9e55701e7c4fe180a21d1934298240` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:58.332817 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ddf69-ec56-49ee-925a-421767895344 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e9e55701e7c4fe180a21d1934298240)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:58.334787 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ddf69-ec56-49ee-925a-421767895344 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:58.382312 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-345ddf69-ec56-49ee-925a-421767895344 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e9e55701e7c4fe180a21d1934298240', 'request_id': 'req-345ddf69-ec56-49ee-925a-421767895344'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:58.388158 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-345ddf69-ec56-49ee-925a-421767895344 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e9e55701e7c4fe180a21d1934298240', 'request_id': 'req-345ddf69-ec56-49ee-925a-421767895344'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:58.395229 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-345ddf69-ec56-49ee-925a-421767895344 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e9e55701e7c4fe180a21d1934298240', 'request_id': 'req-345ddf69-ec56-49ee-925a-421767895344'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:58.408929 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-345ddf69-ec56-49ee-925a-421767895344 admin admin] The token cache is being invalidate because project 5e9e55701e7c4fe180a21d1934298240 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:07:58.409238 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-345ddf69-ec56-49ee-925a-421767895344 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:58.409929 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-345ddf69-ec56-49ee-925a-421767895344 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:58.411643 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1392/2794] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:07:58 2022] DELETE /identity/v3/projects/5e9e55701e7c4fe180a21d1934298240 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:07:58.415894 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a22cb3-ce43-45df-9f68-b3f7047893a8 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:58.418056 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-95a22cb3-ce43-45df-9f68-b3f7047893a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:58.433316 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a22cb3-ce43-45df-9f68-b3f7047893a8 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:58.496864 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ed2d5-23b2-42ee-9734-7f4a4d28262b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:58.497746 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ed2d5-23b2-42ee-9734-7f4a4d28262b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:58.498059 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ed2d5-23b2-42ee-9734-7f4a4d28262b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:58.498343 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ed2d5-23b2-42ee-9734-7f4a4d28262b admin admin] PATH_INFO: `/v3/users/7dac312060bd47d9869f3502ced1dda0` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:58.500075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ed2d5-23b2-42ee-9734-7f4a4d28262b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dac312060bd47d9869f3502ced1dda0)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:58.502045 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ed2d5-23b2-42ee-9734-7f4a4d28262b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:58.554250 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-ea4ed2d5-23b2-42ee-9734-7f4a4d28262b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dac312060bd47d9869f3502ced1dda0'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:58.560527 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:07:58.569510 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-ea4ed2d5-23b2-42ee-9734-7f4a4d28262b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dac312060bd47d9869f3502ced1dda0'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:58.578142 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-ea4ed2d5-23b2-42ee-9734-7f4a4d28262b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dac312060bd47d9869f3502ced1dda0'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:58.578763 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-ea4ed2d5-23b2-42ee-9734-7f4a4d28262b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dac312060bd47d9869f3502ced1dda0'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:58.579375 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-ea4ed2d5-23b2-42ee-9734-7f4a4d28262b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dac312060bd47d9869f3502ced1dda0'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:58.592376 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1403/2795] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:07:58 2022] DELETE /identity/v3/users/7dac312060bd47d9869f3502ced1dda0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:07:58.775013 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ddf69-ec56-49ee-925a-421767895344 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:58.776718 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-345ddf69-ec56-49ee-925a-421767895344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:58.799475 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ddf69-ec56-49ee-925a-421767895344 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:58.824869 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:07:58.872508 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9327bc7-bef3-4de1-8222-aef46c004b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:58.873416 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9327bc7-bef3-4de1-8222-aef46c004b6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:58.873698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9327bc7-bef3-4de1-8222-aef46c004b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:58.874501 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9327bc7-bef3-4de1-8222-aef46c004b6a admin admin] PATH_INFO: `/v3/projects/34062b9020a74975b3a0eb31030e4fa6` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:58.875852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9327bc7-bef3-4de1-8222-aef46c004b6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=34062b9020a74975b3a0eb31030e4fa6)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:58.877823 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9327bc7-bef3-4de1-8222-aef46c004b6a admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:58.939650 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c9327bc7-bef3-4de1-8222-aef46c004b6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34062b9020a74975b3a0eb31030e4fa6', 'request_id': 'req-c9327bc7-bef3-4de1-8222-aef46c004b6a'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:58.945626 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c9327bc7-bef3-4de1-8222-aef46c004b6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34062b9020a74975b3a0eb31030e4fa6', 'request_id': 'req-c9327bc7-bef3-4de1-8222-aef46c004b6a'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:58.952782 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c9327bc7-bef3-4de1-8222-aef46c004b6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34062b9020a74975b3a0eb31030e4fa6', 'request_id': 'req-c9327bc7-bef3-4de1-8222-aef46c004b6a'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:58.969852 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c9327bc7-bef3-4de1-8222-aef46c004b6a admin admin] The token cache is being invalidate because project 34062b9020a74975b3a0eb31030e4fa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:07:58.970363 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c9327bc7-bef3-4de1-8222-aef46c004b6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:58.971148 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-c9327bc7-bef3-4de1-8222-aef46c004b6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:07:58.972898 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1393/2796] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:07:58 2022] DELETE /identity/v3/projects/34062b9020a74975b3a0eb31030e4fa6 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:08:08.070951 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774b132-d8ba-4b6a-b25c-9dac402d9342 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:08:08.071809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774b132-d8ba-4b6a-b25c-9dac402d9342 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:08:08.071809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774b132-d8ba-4b6a-b25c-9dac402d9342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:08:08.091054 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-e774b132-d8ba-4b6a-b25c-9dac402d9342 None None] MFA Rules not processed for user `b2935579c8554bed8b1e211c47548ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:08:08.092424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e774b132-d8ba-4b6a-b25c-9dac402d9342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:08:08.154548 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1404/2797] 10.222.0.78 () {58 vars in 1009 bytes} [Wed May 25 15:08:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:08:08.169028 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9327bc7-bef3-4de1-8222-aef46c004b6a admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:08:08.171419 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-c9327bc7-bef3-4de1-8222-aef46c004b6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:08:08.278001 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9327bc7-bef3-4de1-8222-aef46c004b6a admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:08:08.284744 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df891ca-9fb7-4aca-973d-4a1bac3cf77a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe4a2d106d84fc9ac65c046ba675353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:08:08.285513 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df891ca-9fb7-4aca-973d-4a1bac3cf77a service swift] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:08:08.285855 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df891ca-9fb7-4aca-973d-4a1bac3cf77a service swift] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:08:08.286218 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df891ca-9fb7-4aca-973d-4a1bac3cf77a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:08:08.297164 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df891ca-9fb7-4aca-973d-4a1bac3cf77a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:08:08.299198 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df891ca-9fb7-4aca-973d-4a1bac3cf77a service swift] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:08:08.309232 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1394/2798] 10.222.0.78 () {60 vars in 1384 bytes} [Wed May 25 15:08:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 15:08:10.038762 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774b132-d8ba-4b6a-b25c-9dac402d9342 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:08:10.040356 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e774b132-d8ba-4b6a-b25c-9dac402d9342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:08:10.142222 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774b132-d8ba-4b6a-b25c-9dac402d9342 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:08:10.149584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a99555-fb66-4fc1-a4c1-47412a6e8a3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:08:10.150604 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a99555-fb66-4fc1-a4c1-47412a6e8a3e service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:08:10.151079 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a99555-fb66-4fc1-a4c1-47412a6e8a3e service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:08:10.151477 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a99555-fb66-4fc1-a4c1-47412a6e8a3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:08:10.154127 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-39a99555-fb66-4fc1-a4c1-47412a6e8a3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:08:10.169743 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a99555-fb66-4fc1-a4c1-47412a6e8a3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:08:10.172846 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a99555-fb66-4fc1-a4c1-47412a6e8a3e service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:08:10.193492 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1405/2799] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:08:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:08:18.305741 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:08:20.180657 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=69,keystone:DELETE=2,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:08:22.731051 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f02f10-de5e-46d9-8d82-7b93bc2ea6f6 None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:08:22.731619 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f02f10-de5e-46d9-8d82-7b93bc2ea6f6 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:08:22.731619 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f02f10-de5e-46d9-8d82-7b93bc2ea6f6 None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:08:22.733128 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1395/2800] 10.222.0.78 () {54 vars in 921 bytes} [Wed May 25 15:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:08:22.739771 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b03a93-2495-42ee-85c0-e5f795333ea6 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:08:22.740256 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b03a93-2495-42ee-85c0-e5f795333ea6 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:08:22.740387 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b03a93-2495-42ee-85c0-e5f795333ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:08:22.769161 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-59b03a93-2495-42ee-85c0-e5f795333ea6 None None] MFA Rules not processed for user `e34e1d7ffbac49b0bde45829af0c6925`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:08:22.770333 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-59b03a93-2495-42ee-85c0-e5f795333ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:08:22.779691 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1406/2801] 10.222.0.78 () {58 vars in 1018 bytes} [Wed May 25 15:08:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:08:22.792291 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f02f10-de5e-46d9-8d82-7b93bc2ea6f6 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:08:22.794038 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-92f02f10-de5e-46d9-8d82-7b93bc2ea6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:08:22.883878 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f02f10-de5e-46d9-8d82-7b93bc2ea6f6 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:08:22.887967 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b66a3a-6911-4879-9345-ccefb1958c53 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88cbd76f70d447c4ba92777b2c09821d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:08:22.888646 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b66a3a-6911-4879-9345-ccefb1958c53 service glance] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:08:22.888928 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b66a3a-6911-4879-9345-ccefb1958c53 service glance] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:08:22.889210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b66a3a-6911-4879-9345-ccefb1958c53 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:08:22.900606 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b66a3a-6911-4879-9345-ccefb1958c53 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:08:22.902101 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b66a3a-6911-4879-9345-ccefb1958c53 service glance] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:08:22.913373 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1396/2802] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:08:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:08:22.979274 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d138644-9e5f-4a13-b4db-b009164d147f None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:08:22.979596 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d138644-9e5f-4a13-b4db-b009164d147f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:08:22.979871 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d138644-9e5f-4a13-b4db-b009164d147f None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:08:22.981246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1407/2803] 10.222.0.78 () {54 vars in 921 bytes} [Wed May 25 15:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:08:22.986700 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246a748-934f-46e4-9a63-f0afea6e3fda None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:08:22.986989 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246a748-934f-46e4-9a63-f0afea6e3fda None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:08:22.987267 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246a748-934f-46e4-9a63-f0afea6e3fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:08:23.035789 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-d246a748-934f-46e4-9a63-f0afea6e3fda None None] MFA Rules not processed for user `70ad51a8ad68440da6618891e2e08cb8`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:08:23.036925 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d246a748-934f-46e4-9a63-f0afea6e3fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:08:23.087218 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1397/2804] 10.222.0.78 () {58 vars in 1018 bytes} [Wed May 25 15:08:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:08:23.099593 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d138644-9e5f-4a13-b4db-b009164d147f None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:08:23.101778 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7d138644-9e5f-4a13-b4db-b009164d147f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:08:23.117008 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d138644-9e5f-4a13-b4db-b009164d147f None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:08:23.120621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5842b0-8706-42fd-80ec-6b2a2367ef01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:08:23.121265 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5842b0-8706-42fd-80ec-6b2a2367ef01 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:08:23.121552 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5842b0-8706-42fd-80ec-6b2a2367ef01 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:08:23.121831 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5842b0-8706-42fd-80ec-6b2a2367ef01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:08:23.129947 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5842b0-8706-42fd-80ec-6b2a2367ef01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:08:23.131426 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5842b0-8706-42fd-80ec-6b2a2367ef01 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:08:23.139706 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1408/2805] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:08:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:08:23.226382 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350df338-ea64-454d-a17f-fdb161e733ae None None] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:08:23.226671 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350df338-ea64-454d-a17f-fdb161e733ae None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:08:23.226952 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350df338-ea64-454d-a17f-fdb161e733ae None None] PATH_INFO: `/` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:08:23.228415 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1398/2806] 10.222.0.78 () {54 vars in 921 bytes} [Wed May 25 15:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:08:23.234262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d4c02-8339-4cee-b2fc-c61f9e0b0388 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:08:23.234603 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d4c02-8339-4cee-b2fc-c61f9e0b0388 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:08:23.235106 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d4c02-8339-4cee-b2fc-c61f9e0b0388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:08:23.254507 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-a47d4c02-8339-4cee-b2fc-c61f9e0b0388 None None] MFA Rules not processed for user `e34e1d7ffbac49b0bde45829af0c6925`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:08:23.255705 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a47d4c02-8339-4cee-b2fc-c61f9e0b0388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:08:23.265839 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1409/2807] 10.222.0.78 () {58 vars in 1018 bytes} [Wed May 25 15:08:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:08:23.276501 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350df338-ea64-454d-a17f-fdb161e733ae None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:08:23.288406 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350df338-ea64-454d-a17f-fdb161e733ae None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:08:23.291998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609e428-d485-45e1-9763-04742623d6eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:08:23.292667 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609e428-d485-45e1-9763-04742623d6eb service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:08:23.292947 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609e428-d485-45e1-9763-04742623d6eb service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:08:23.293223 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609e428-d485-45e1-9763-04742623d6eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:08:23.303483 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609e428-d485-45e1-9763-04742623d6eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:08:23.304915 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609e428-d485-45e1-9763-04742623d6eb service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:08:23.315987 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1399/2808] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:08:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:08:25.183254 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cea6c4-f8ff-49c5-b5f8-1ad12e7355fd None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:08:25.183733 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cea6c4-f8ff-49c5-b5f8-1ad12e7355fd None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:08:25.183852 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cea6c4-f8ff-49c5-b5f8-1ad12e7355fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:08:25.201908 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-08cea6c4-f8ff-49c5-b5f8-1ad12e7355fd None None] MFA Rules not processed for user `b2935579c8554bed8b1e211c47548ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:08:25.203702 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-08cea6c4-f8ff-49c5-b5f8-1ad12e7355fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:08:25.211567 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1410/2809] 10.222.0.78 () {58 vars in 1009 bytes} [Wed May 25 15:08:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:08:25.223309 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609e428-d485-45e1-9763-04742623d6eb service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:08:25.238096 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609e428-d485-45e1-9763-04742623d6eb service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:08:25.243376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d928c8f-e571-40f6-8b35-df8b1cefa621 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe4a2d106d84fc9ac65c046ba675353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:08:25.244019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d928c8f-e571-40f6-8b35-df8b1cefa621 service swift] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:08:25.244301 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d928c8f-e571-40f6-8b35-df8b1cefa621 service swift] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:08:25.244580 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d928c8f-e571-40f6-8b35-df8b1cefa621 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:08:25.253741 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d928c8f-e571-40f6-8b35-df8b1cefa621 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:08:25.255416 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d928c8f-e571-40f6-8b35-df8b1cefa621 service swift] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:08:25.264124 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1400/2810] 10.222.0.78 () {60 vars in 1384 bytes} [Wed May 25 15:08:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 15:08:35.196546 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=19 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:08:35.260260 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=81 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:08:41.781244 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a10361-4acc-480c-b685-106831497b27 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:08:41.781729 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a10361-4acc-480c-b685-106831497b27 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:08:41.781828 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a10361-4acc-480c-b685-106831497b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:08:41.805587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-e5a10361-4acc-480c-b685-106831497b27 None None] MFA Rules not processed for user `b2935579c8554bed8b1e211c47548ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:08:41.806897 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e5a10361-4acc-480c-b685-106831497b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:08:41.815850 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1411/2811] 10.222.0.78 () {58 vars in 967 bytes} [Wed May 25 15:08:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:08:41.829387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d928c8f-e571-40f6-8b35-df8b1cefa621 service swift] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:08:41.840308 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d928c8f-e571-40f6-8b35-df8b1cefa621 service swift] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:08:41.845561 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593891fa-cff2-4cff-992d-177ec9f56451 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe4a2d106d84fc9ac65c046ba675353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:08:41.846242 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593891fa-cff2-4cff-992d-177ec9f56451 service swift] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:08:41.846528 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593891fa-cff2-4cff-992d-177ec9f56451 service swift] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:08:41.846804 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593891fa-cff2-4cff-992d-177ec9f56451 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:08:41.855155 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593891fa-cff2-4cff-992d-177ec9f56451 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:08:41.856726 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593891fa-cff2-4cff-992d-177ec9f56451 service swift] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:08:41.865518 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1401/2812] 10.222.0.78 () {60 vars in 1384 bytes} [Wed May 25 15:08:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 15:08:51.799819 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:08:51.862189 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=6 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:09:31.144838 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a10361-4acc-480c-b685-106831497b27 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:31.154860 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a10361-4acc-480c-b685-106831497b27 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:31.158524 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15e63e-913f-44fd-955f-ca80027c75de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:31.159101 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15e63e-913f-44fd-955f-ca80027c75de service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:31.159387 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15e63e-913f-44fd-955f-ca80027c75de service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:31.159685 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15e63e-913f-44fd-955f-ca80027c75de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:31.162111 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ad15e63e-913f-44fd-955f-ca80027c75de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:31.227702 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15e63e-913f-44fd-955f-ca80027c75de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:31.229202 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15e63e-913f-44fd-955f-ca80027c75de service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:31.239052 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1412/2813] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:09:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:09:34.184365 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593891fa-cff2-4cff-992d-177ec9f56451 service swift] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:34.185891 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-593891fa-cff2-4cff-992d-177ec9f56451 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:34.262275 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15e63e-913f-44fd-955f-ca80027c75de service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:34.263699 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ad15e63e-913f-44fd-955f-ca80027c75de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:34.279199 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15e63e-913f-44fd-955f-ca80027c75de service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:34.284287 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77192acf-5181-4733-82cf-ad5866d6d346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:34.285271 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77192acf-5181-4733-82cf-ad5866d6d346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:34.285626 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77192acf-5181-4733-82cf-ad5866d6d346 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:34.286575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77192acf-5181-4733-82cf-ad5866d6d346 admin admin] PATH_INFO: `/v3/users/34b1591b0eb944ed98c42d35a03c1ff9` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:34.287849 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77192acf-5181-4733-82cf-ad5866d6d346 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34b1591b0eb944ed98c42d35a03c1ff9)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:34.289907 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77192acf-5181-4733-82cf-ad5866d6d346 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:34.319042 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593891fa-cff2-4cff-992d-177ec9f56451 service swift] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:34.325566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068f087-7b41-4961-b04d-7a91d3b7cecc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63789c0a113447ff90933d447cf5bac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:34.326371 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068f087-7b41-4961-b04d-7a91d3b7cecc service placement] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:34.326927 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068f087-7b41-4961-b04d-7a91d3b7cecc service placement] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:34.327347 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068f087-7b41-4961-b04d-7a91d3b7cecc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:34.329252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-8068f087-7b41-4961-b04d-7a91d3b7cecc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:34.342214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-77192acf-5181-4733-82cf-ad5866d6d346 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34b1591b0eb944ed98c42d35a03c1ff9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:34.342652 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8068f087-7b41-4961-b04d-7a91d3b7cecc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:34.344259 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8068f087-7b41-4961-b04d-7a91d3b7cecc service placement] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:34.356341 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1402/2814] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:09:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:09:34.360329 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-77192acf-5181-4733-82cf-ad5866d6d346 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34b1591b0eb944ed98c42d35a03c1ff9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:34.369745 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-77192acf-5181-4733-82cf-ad5866d6d346 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34b1591b0eb944ed98c42d35a03c1ff9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:34.370363 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-77192acf-5181-4733-82cf-ad5866d6d346 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34b1591b0eb944ed98c42d35a03c1ff9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:34.371140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-77192acf-5181-4733-82cf-ad5866d6d346 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34b1591b0eb944ed98c42d35a03c1ff9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:34.386666 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1413/2815] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:09:34 2022] DELETE /identity/v3/users/34b1591b0eb944ed98c42d35a03c1ff9 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:09:34.574398 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068f087-7b41-4961-b04d-7a91d3b7cecc service placement] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:34.588272 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068f087-7b41-4961-b04d-7a91d3b7cecc service placement] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:34.654873 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c3ee3-6175-43a7-9fdc-2daeeb2b603a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:34.655745 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c3ee3-6175-43a7-9fdc-2daeeb2b603a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:34.656030 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c3ee3-6175-43a7-9fdc-2daeeb2b603a admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:34.656303 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c3ee3-6175-43a7-9fdc-2daeeb2b603a admin admin] PATH_INFO: `/v3/projects/2b586a13e82f4177a7180f81acf675fd` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:34.657492 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c3ee3-6175-43a7-9fdc-2daeeb2b603a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b586a13e82f4177a7180f81acf675fd)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:34.659442 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c3ee3-6175-43a7-9fdc-2daeeb2b603a admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:34.706514 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-4d9c3ee3-6175-43a7-9fdc-2daeeb2b603a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b586a13e82f4177a7180f81acf675fd', 'request_id': 'req-4d9c3ee3-6175-43a7-9fdc-2daeeb2b603a'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:34.712920 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-4d9c3ee3-6175-43a7-9fdc-2daeeb2b603a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b586a13e82f4177a7180f81acf675fd', 'request_id': 'req-4d9c3ee3-6175-43a7-9fdc-2daeeb2b603a'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:34.719271 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-4d9c3ee3-6175-43a7-9fdc-2daeeb2b603a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b586a13e82f4177a7180f81acf675fd', 'request_id': 'req-4d9c3ee3-6175-43a7-9fdc-2daeeb2b603a'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:34.729120 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:09:34.734144 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-4d9c3ee3-6175-43a7-9fdc-2daeeb2b603a admin admin] The token cache is being invalidate because project 2b586a13e82f4177a7180f81acf675fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:09:34.734700 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-4d9c3ee3-6175-43a7-9fdc-2daeeb2b603a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:34.735611 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-4d9c3ee3-6175-43a7-9fdc-2daeeb2b603a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:34.737979 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1403/2816] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:09:34 2022] DELETE /identity/v3/projects/2b586a13e82f4177a7180f81acf675fd => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:09:34.743014 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86233bb8-f192-4609-8bf7-5ed3349e2eea None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:34.743309 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86233bb8-f192-4609-8bf7-5ed3349e2eea None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:34.743621 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86233bb8-f192-4609-8bf7-5ed3349e2eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:34.792413 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-86233bb8-f192-4609-8bf7-5ed3349e2eea None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:09:34.793656 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-86233bb8-f192-4609-8bf7-5ed3349e2eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:34.840622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:09:34.861155 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1414/2817] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:09:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:09:34.870320 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e387f1-5432-4ece-a8fb-7b72f16c706c None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:34.870609 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e387f1-5432-4ece-a8fb-7b72f16c706c None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:34.870886 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e387f1-5432-4ece-a8fb-7b72f16c706c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:34.889977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-c0e387f1-5432-4ece-a8fb-7b72f16c706c None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:09:34.891233 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-c0e387f1-5432-4ece-a8fb-7b72f16c706c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:34.899850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1404/2818] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:09:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:09:34.904230 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86233bb8-f192-4609-8bf7-5ed3349e2eea None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:34.914158 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86233bb8-f192-4609-8bf7-5ed3349e2eea None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:34.918856 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626a279-3d85-457d-b002-10516960320e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:34.919519 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626a279-3d85-457d-b002-10516960320e admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:34.919809 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626a279-3d85-457d-b002-10516960320e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:34.920091 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626a279-3d85-457d-b002-10516960320e admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:34.920816 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b626a279-3d85-457d-b002-10516960320e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:09:34.921128 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b626a279-3d85-457d-b002-10516960320e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:34.922846 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b626a279-3d85-457d-b002-10516960320e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:34.934997 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1415/2819] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 15:09:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:09:34.939302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e387f1-5432-4ece-a8fb-7b72f16c706c None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:34.949366 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e387f1-5432-4ece-a8fb-7b72f16c706c None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:34.954079 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee795f0-dd70-421e-9c91-fc674ef12448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:34.954735 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee795f0-dd70-421e-9c91-fc674ef12448 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:34.955019 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee795f0-dd70-421e-9c91-fc674ef12448 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:34.955297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee795f0-dd70-421e-9c91-fc674ef12448 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:34.955998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee795f0-dd70-421e-9c91-fc674ef12448 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:34.958668 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee795f0-dd70-421e-9c91-fc674ef12448 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:34.984726 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1405/2820] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:09:34 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:09:34.989294 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626a279-3d85-457d-b002-10516960320e admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:34.999506 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626a279-3d85-457d-b002-10516960320e admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:35.059949 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f969124-333f-41bb-9293-901c4ce95da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:35.060627 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f969124-333f-41bb-9293-901c4ce95da9 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:35.060913 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f969124-333f-41bb-9293-901c4ce95da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:35.061194 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f969124-333f-41bb-9293-901c4ce95da9 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:35.061888 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f969124-333f-41bb-9293-901c4ce95da9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:35.063902 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f969124-333f-41bb-9293-901c4ce95da9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:35.085605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1416/2821] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:09:34 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:09:35.089901 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee795f0-dd70-421e-9c91-fc674ef12448 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:35.100395 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee795f0-dd70-421e-9c91-fc674ef12448 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:35.105305 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73478b6-ed3a-4d19-98ec-aa5faa8f2c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:35.105966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73478b6-ed3a-4d19-98ec-aa5faa8f2c48 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:35.106283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73478b6-ed3a-4d19-98ec-aa5faa8f2c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:35.106557 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73478b6-ed3a-4d19-98ec-aa5faa8f2c48 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:35.107280 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73478b6-ed3a-4d19-98ec-aa5faa8f2c48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:35.108891 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73478b6-ed3a-4d19-98ec-aa5faa8f2c48 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:35.121194 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1406/2822] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:09:35.125273 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f969124-333f-41bb-9293-901c4ce95da9 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:35.137041 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f969124-333f-41bb-9293-901c4ce95da9 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:35.142396 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919472b-7dd7-460a-93b5-fb51664e0f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:35.143072 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919472b-7dd7-460a-93b5-fb51664e0f9e admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:35.143358 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919472b-7dd7-460a-93b5-fb51664e0f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:35.143633 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919472b-7dd7-460a-93b5-fb51664e0f9e admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:35.144372 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8919472b-7dd7-460a-93b5-fb51664e0f9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:35.146039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8919472b-7dd7-460a-93b5-fb51664e0f9e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:35.156537 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1417/2823] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:09:35.160759 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73478b6-ed3a-4d19-98ec-aa5faa8f2c48 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:35.170937 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73478b6-ed3a-4d19-98ec-aa5faa8f2c48 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:35.175612 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3b62d-1014-43c0-894e-a9153a123a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:35.177001 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3b62d-1014-43c0-894e-a9153a123a39 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:35.177281 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3b62d-1014-43c0-894e-a9153a123a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:35.177552 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3b62d-1014-43c0-894e-a9153a123a39 admin admin] PATH_INFO: `/v3/projects/dedabf0e546648a581d7c0949303e212/users/221a6ad4936a4d58ad5c552ad439e5c8/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:35.195540 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e3b62d-1014-43c0-894e-a9153a123a39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dedabf0e546648a581d7c0949303e212, user_id=221a6ad4936a4d58ad5c552ad439e5c8, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:35.198287 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e3b62d-1014-43c0-894e-a9153a123a39 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:35.213031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1407/2824] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:09:35 2022] PUT /identity/v3/projects/dedabf0e546648a581d7c0949303e212/users/221a6ad4936a4d58ad5c552ad439e5c8/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:09:35.224045 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7d418-fd38-4f5c-ae23-71630cac1394 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:35.224333 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7d418-fd38-4f5c-ae23-71630cac1394 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:35.224641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7d418-fd38-4f5c-ae23-71630cac1394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:35.254301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-e9b7d418-fd38-4f5c-ae23-71630cac1394 None None] MFA Rules not processed for user `221a6ad4936a4d58ad5c552ad439e5c8`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:09:35.255634 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e9b7d418-fd38-4f5c-ae23-71630cac1394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:35.332984 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1418/2825] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:09:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:09:35.337701 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3b62d-1014-43c0-894e-a9153a123a39 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:35.403548 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3b62d-1014-43c0-894e-a9153a123a39 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:35.408734 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8475e-4ecd-4fbe-821e-af39f96af95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:35.409413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8475e-4ecd-4fbe-821e-af39f96af95e admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:35.409696 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8475e-4ecd-4fbe-821e-af39f96af95e admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:35.409977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8475e-4ecd-4fbe-821e-af39f96af95e admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:35.411283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb8475e-4ecd-4fbe-821e-af39f96af95e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:35.413385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb8475e-4ecd-4fbe-821e-af39f96af95e admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:35.441631 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1408/2826] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:09:35 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:09:35.445770 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7d418-fd38-4f5c-ae23-71630cac1394 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:35.456587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7d418-fd38-4f5c-ae23-71630cac1394 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:35.496022 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:09:35.519618 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e343cc-0e8c-4bfe-85cb-93bd01af7cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:35.520209 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e343cc-0e8c-4bfe-85cb-93bd01af7cba admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:35.520631 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e343cc-0e8c-4bfe-85cb-93bd01af7cba admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:35.520817 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e343cc-0e8c-4bfe-85cb-93bd01af7cba admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:35.521756 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e343cc-0e8c-4bfe-85cb-93bd01af7cba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:35.523469 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e343cc-0e8c-4bfe-85cb-93bd01af7cba admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:35.546815 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1419/2827] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:09:35 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:09:35.550899 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8475e-4ecd-4fbe-821e-af39f96af95e admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:35.561925 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8475e-4ecd-4fbe-821e-af39f96af95e admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:35.567613 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043d34f-4ed2-4472-aca5-e8ccc5d27240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:35.568500 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043d34f-4ed2-4472-aca5-e8ccc5d27240 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:35.568959 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043d34f-4ed2-4472-aca5-e8ccc5d27240 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:35.569426 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043d34f-4ed2-4472-aca5-e8ccc5d27240 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:35.570436 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043d34f-4ed2-4472-aca5-e8ccc5d27240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:35.573628 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043d34f-4ed2-4472-aca5-e8ccc5d27240 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:35.596059 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1409/2828] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:09:35.601036 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e343cc-0e8c-4bfe-85cb-93bd01af7cba admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:35.611737 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e343cc-0e8c-4bfe-85cb-93bd01af7cba admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:35.616956 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c2980-6cc0-4419-bc9b-6fb4baa7df06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:35.617884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c2980-6cc0-4419-bc9b-6fb4baa7df06 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:35.618199 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c2980-6cc0-4419-bc9b-6fb4baa7df06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:35.618496 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c2980-6cc0-4419-bc9b-6fb4baa7df06 admin admin] PATH_INFO: `/v3/projects/9c40514f952746a0a15eedea713a613e/users/cc415814e36c49219ac0f039238bb7cc/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:35.635884 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c2980-6cc0-4419-bc9b-6fb4baa7df06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c40514f952746a0a15eedea713a613e, user_id=cc415814e36c49219ac0f039238bb7cc, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:35.638657 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c2980-6cc0-4419-bc9b-6fb4baa7df06 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:35.651967 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1420/2829] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:09:35 2022] PUT /identity/v3/projects/9c40514f952746a0a15eedea713a613e/users/cc415814e36c49219ac0f039238bb7cc/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:09:35.656014 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043d34f-4ed2-4472-aca5-e8ccc5d27240 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:35.667062 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043d34f-4ed2-4472-aca5-e8ccc5d27240 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:35.724864 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499cf47-d4c4-46ac-b648-044d41ad3185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:35.725538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499cf47-d4c4-46ac-b648-044d41ad3185 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:35.725827 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499cf47-d4c4-46ac-b648-044d41ad3185 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:35.726129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499cf47-d4c4-46ac-b648-044d41ad3185 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:35.726854 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b499cf47-d4c4-46ac-b648-044d41ad3185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:35.728452 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b499cf47-d4c4-46ac-b648-044d41ad3185 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:35.738407 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1410/2830] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:09:35.742740 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c2980-6cc0-4419-bc9b-6fb4baa7df06 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:35.753180 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c2980-6cc0-4419-bc9b-6fb4baa7df06 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:35.757720 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466caa98-3999-4842-be5d-eabd9885fa4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:35.758705 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466caa98-3999-4842-be5d-eabd9885fa4f admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:35.758905 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466caa98-3999-4842-be5d-eabd9885fa4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:35.759179 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466caa98-3999-4842-be5d-eabd9885fa4f admin admin] PATH_INFO: `/v3/projects/9c40514f952746a0a15eedea713a613e/users/cc415814e36c49219ac0f039238bb7cc/roles/e566b37cd92f467198b5dc40f183effc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:35.761412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466caa98-3999-4842-be5d-eabd9885fa4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c40514f952746a0a15eedea713a613e, user_id=cc415814e36c49219ac0f039238bb7cc, role_id=e566b37cd92f467198b5dc40f183effc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:35.764077 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466caa98-3999-4842-be5d-eabd9885fa4f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:35.774829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1421/2831] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:09:35 2022] PUT /identity/v3/projects/9c40514f952746a0a15eedea713a613e/users/cc415814e36c49219ac0f039238bb7cc/roles/e566b37cd92f467198b5dc40f183effc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:09:35.784818 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6a4f3-49e3-4517-becd-4555dcc65bac None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:35.785184 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6a4f3-49e3-4517-becd-4555dcc65bac None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:35.785535 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6a4f3-49e3-4517-becd-4555dcc65bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:35.798816 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:09:35.814782 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-07d6a4f3-49e3-4517-becd-4555dcc65bac None None] MFA Rules not processed for user `cc415814e36c49219ac0f039238bb7cc`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:09:35.816006 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-07d6a4f3-49e3-4517-becd-4555dcc65bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:35.885067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1411/2832] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:09:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:09:35.898832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466caa98-3999-4842-be5d-eabd9885fa4f admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:35.900913 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-466caa98-3999-4842-be5d-eabd9885fa4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:35.981663 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466caa98-3999-4842-be5d-eabd9885fa4f admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:35.985401 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b94769-6e62-4f5d-adbe-6945f5689ed5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:35.986080 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b94769-6e62-4f5d-adbe-6945f5689ed5 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:35.986366 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b94769-6e62-4f5d-adbe-6945f5689ed5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:35.986646 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b94769-6e62-4f5d-adbe-6945f5689ed5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:35.996047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b94769-6e62-4f5d-adbe-6945f5689ed5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:35.997491 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b94769-6e62-4f5d-adbe-6945f5689ed5 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:36.007750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1422/2833] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:09:35 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:09:36.072746 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6a4f3-49e3-4517-becd-4555dcc65bac None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:36.135297 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6a4f3-49e3-4517-becd-4555dcc65bac None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:36.140850 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eccdd5-9a89-4674-95f0-3225c4aaf936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:36.141736 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eccdd5-9a89-4674-95f0-3225c4aaf936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:36.142031 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eccdd5-9a89-4674-95f0-3225c4aaf936 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:36.142344 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eccdd5-9a89-4674-95f0-3225c4aaf936 admin admin] PATH_INFO: `/v3/users/221a6ad4936a4d58ad5c552ad439e5c8` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:36.143592 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eccdd5-9a89-4674-95f0-3225c4aaf936 admin admin] RBAC: Authorizing `identity:delete_user(user_id=221a6ad4936a4d58ad5c552ad439e5c8)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:36.146164 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eccdd5-9a89-4674-95f0-3225c4aaf936 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:36.195336 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f0eccdd5-9a89-4674-95f0-3225c4aaf936 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '221a6ad4936a4d58ad5c552ad439e5c8'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:36.208353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f0eccdd5-9a89-4674-95f0-3225c4aaf936 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '221a6ad4936a4d58ad5c552ad439e5c8'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:36.215844 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f0eccdd5-9a89-4674-95f0-3225c4aaf936 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '221a6ad4936a4d58ad5c552ad439e5c8'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:36.216460 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f0eccdd5-9a89-4674-95f0-3225c4aaf936 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '221a6ad4936a4d58ad5c552ad439e5c8'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:36.217062 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f0eccdd5-9a89-4674-95f0-3225c4aaf936 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '221a6ad4936a4d58ad5c552ad439e5c8'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:36.230228 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1412/2834] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:09:36 2022] DELETE /identity/v3/users/221a6ad4936a4d58ad5c552ad439e5c8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:09:36.234743 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da2278-e358-4b30-89f5-497f928533af None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:36.235038 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da2278-e358-4b30-89f5-497f928533af None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:36.235323 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da2278-e358-4b30-89f5-497f928533af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:36.253301 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-20da2278-e358-4b30-89f5-497f928533af None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:09:36.254564 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-20da2278-e358-4b30-89f5-497f928533af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:36.326944 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1423/2835] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:09:36.339501 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eccdd5-9a89-4674-95f0-3225c4aaf936 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:36.341190 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f0eccdd5-9a89-4674-95f0-3225c4aaf936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:36.355538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:09:36.402464 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eccdd5-9a89-4674-95f0-3225c4aaf936 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:36.406230 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fcdcf-b1bc-407d-aa6d-d5fb2bd7c734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:36.406883 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fcdcf-b1bc-407d-aa6d-d5fb2bd7c734 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:36.407164 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fcdcf-b1bc-407d-aa6d-d5fb2bd7c734 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:36.407437 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fcdcf-b1bc-407d-aa6d-d5fb2bd7c734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:36.417355 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911fcdcf-b1bc-407d-aa6d-d5fb2bd7c734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:36.418829 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911fcdcf-b1bc-407d-aa6d-d5fb2bd7c734 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:36.428814 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1413/2836] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:09:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:09:36.698625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da2278-e358-4b30-89f5-497f928533af None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:36.700096 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-20da2278-e358-4b30-89f5-497f928533af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:36.713911 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da2278-e358-4b30-89f5-497f928533af None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:36.718728 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a806567-21f8-42a7-85d7-6ffc37f317b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:36.719645 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a806567-21f8-42a7-85d7-6ffc37f317b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:36.720111 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a806567-21f8-42a7-85d7-6ffc37f317b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:36.720392 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a806567-21f8-42a7-85d7-6ffc37f317b5 admin admin] PATH_INFO: `/v3/projects/dedabf0e546648a581d7c0949303e212` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:36.721622 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a806567-21f8-42a7-85d7-6ffc37f317b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dedabf0e546648a581d7c0949303e212)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:36.723604 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a806567-21f8-42a7-85d7-6ffc37f317b5 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:36.735189 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:09:36.773793 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5a806567-21f8-42a7-85d7-6ffc37f317b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dedabf0e546648a581d7c0949303e212', 'request_id': 'req-5a806567-21f8-42a7-85d7-6ffc37f317b5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:36.779357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5a806567-21f8-42a7-85d7-6ffc37f317b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dedabf0e546648a581d7c0949303e212', 'request_id': 'req-5a806567-21f8-42a7-85d7-6ffc37f317b5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:36.786094 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5a806567-21f8-42a7-85d7-6ffc37f317b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dedabf0e546648a581d7c0949303e212', 'request_id': 'req-5a806567-21f8-42a7-85d7-6ffc37f317b5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:36.799297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5a806567-21f8-42a7-85d7-6ffc37f317b5 admin admin] The token cache is being invalidate because project dedabf0e546648a581d7c0949303e212 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:09:36.799594 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5a806567-21f8-42a7-85d7-6ffc37f317b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:36.800284 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5a806567-21f8-42a7-85d7-6ffc37f317b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:36.801928 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1424/2837] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:09:36 2022] DELETE /identity/v3/projects/dedabf0e546648a581d7c0949303e212 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:09:36.806254 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fcdcf-b1bc-407d-aa6d-d5fb2bd7c734 service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:36.816451 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fcdcf-b1bc-407d-aa6d-d5fb2bd7c734 service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:36.873582 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d41cbd-d3de-4068-ba50-5c6b809beb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:36.874412 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d41cbd-d3de-4068-ba50-5c6b809beb8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:36.874691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d41cbd-d3de-4068-ba50-5c6b809beb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:36.874970 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d41cbd-d3de-4068-ba50-5c6b809beb8d admin admin] PATH_INFO: `/v3/users/cc415814e36c49219ac0f039238bb7cc` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:36.876157 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d41cbd-d3de-4068-ba50-5c6b809beb8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc415814e36c49219ac0f039238bb7cc)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:36.878132 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d41cbd-d3de-4068-ba50-5c6b809beb8d admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:36.927991 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-07d41cbd-d3de-4068-ba50-5c6b809beb8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc415814e36c49219ac0f039238bb7cc'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:36.940136 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-07d41cbd-d3de-4068-ba50-5c6b809beb8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc415814e36c49219ac0f039238bb7cc'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:36.949073 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-07d41cbd-d3de-4068-ba50-5c6b809beb8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc415814e36c49219ac0f039238bb7cc'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:36.949739 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-07d41cbd-d3de-4068-ba50-5c6b809beb8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc415814e36c49219ac0f039238bb7cc'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:36.950392 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-07d41cbd-d3de-4068-ba50-5c6b809beb8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc415814e36c49219ac0f039238bb7cc'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:36.964882 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1414/2838] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:09:36 2022] DELETE /identity/v3/users/cc415814e36c49219ac0f039238bb7cc => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:09:37.265606 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a806567-21f8-42a7-85d7-6ffc37f317b5 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:37.267168 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5a806567-21f8-42a7-85d7-6ffc37f317b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:37.280939 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a806567-21f8-42a7-85d7-6ffc37f317b5 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:37.345566 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9e0bf-82dd-43cb-afcb-f6bfa87719ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:37.346488 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9e0bf-82dd-43cb-afcb-f6bfa87719ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:37.346774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9e0bf-82dd-43cb-afcb-f6bfa87719ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:37.347053 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9e0bf-82dd-43cb-afcb-f6bfa87719ac admin admin] PATH_INFO: `/v3/projects/9c40514f952746a0a15eedea713a613e` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:37.348305 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b9e0bf-82dd-43cb-afcb-f6bfa87719ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c40514f952746a0a15eedea713a613e)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:37.350290 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b9e0bf-82dd-43cb-afcb-f6bfa87719ac admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:37.402977 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-14b9e0bf-82dd-43cb-afcb-f6bfa87719ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c40514f952746a0a15eedea713a613e', 'request_id': 'req-14b9e0bf-82dd-43cb-afcb-f6bfa87719ac'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:37.408625 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-14b9e0bf-82dd-43cb-afcb-f6bfa87719ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c40514f952746a0a15eedea713a613e', 'request_id': 'req-14b9e0bf-82dd-43cb-afcb-f6bfa87719ac'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:37.415448 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-14b9e0bf-82dd-43cb-afcb-f6bfa87719ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c40514f952746a0a15eedea713a613e', 'request_id': 'req-14b9e0bf-82dd-43cb-afcb-f6bfa87719ac'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:37.430531 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-14b9e0bf-82dd-43cb-afcb-f6bfa87719ac admin admin] The token cache is being invalidate because project 9c40514f952746a0a15eedea713a613e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:09:37.430826 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-14b9e0bf-82dd-43cb-afcb-f6bfa87719ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:37.431453 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-14b9e0bf-82dd-43cb-afcb-f6bfa87719ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:09:37.433047 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1425/2839] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:09:37 2022] DELETE /identity/v3/projects/9c40514f952746a0a15eedea713a613e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:09:37.439683 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6716a-d4f2-4e3b-9dbe-244d2ea7772e None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:37.440222 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6716a-d4f2-4e3b-9dbe-244d2ea7772e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:37.440411 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6716a-d4f2-4e3b-9dbe-244d2ea7772e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:37.460730 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-15e6716a-d4f2-4e3b-9dbe-244d2ea7772e None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:09:37.461949 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-15e6716a-d4f2-4e3b-9dbe-244d2ea7772e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:37.492891 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:09:37.529797 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1415/2840] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:09:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:09:37.537871 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbe5fe-ae2c-4469-b9ba-9d31beadd72b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:37.538198 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbe5fe-ae2c-4469-b9ba-9d31beadd72b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:37.538479 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbe5fe-ae2c-4469-b9ba-9d31beadd72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:37.559346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-0fbbe5fe-ae2c-4469-b9ba-9d31beadd72b None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:09:37.560668 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-0fbbe5fe-ae2c-4469-b9ba-9d31beadd72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:37.576364 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1426/2841] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:09:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:09:37.583342 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6716a-d4f2-4e3b-9dbe-244d2ea7772e None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:37.593827 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6716a-d4f2-4e3b-9dbe-244d2ea7772e None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:37.600276 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc8f20-6a74-49f2-93c7-edd2ecc2a645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:37.600504 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc8f20-6a74-49f2-93c7-edd2ecc2a645 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:37.600721 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc8f20-6a74-49f2-93c7-edd2ecc2a645 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:37.600998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc8f20-6a74-49f2-93c7-edd2ecc2a645 admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:37.601739 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc8f20-6a74-49f2-93c7-edd2ecc2a645 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:09:37.602093 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc8f20-6a74-49f2-93c7-edd2ecc2a645 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:37.603710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc8f20-6a74-49f2-93c7-edd2ecc2a645 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:37.617961 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1416/2842] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 15:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:09:37.622826 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbe5fe-ae2c-4469-b9ba-9d31beadd72b None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:37.633687 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbe5fe-ae2c-4469-b9ba-9d31beadd72b None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:37.640913 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b68e77-9723-4f21-8302-0710e48fa24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:37.642048 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b68e77-9723-4f21-8302-0710e48fa24d admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:37.642577 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b68e77-9723-4f21-8302-0710e48fa24d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:37.643072 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b68e77-9723-4f21-8302-0710e48fa24d admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:37.644384 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b68e77-9723-4f21-8302-0710e48fa24d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:37.649046 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b68e77-9723-4f21-8302-0710e48fa24d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:37.674743 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:09:37.690541 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1427/2843] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:09:37 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:09:37.696416 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc8f20-6a74-49f2-93c7-edd2ecc2a645 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:37.707464 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc8f20-6a74-49f2-93c7-edd2ecc2a645 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:37.769569 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce33ab27-4c6f-4e66-8ba4-d6e35aa31d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:37.770819 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33ab27-4c6f-4e66-8ba4-d6e35aa31d50 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:37.771100 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33ab27-4c6f-4e66-8ba4-d6e35aa31d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:37.771373 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33ab27-4c6f-4e66-8ba4-d6e35aa31d50 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:37.772109 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce33ab27-4c6f-4e66-8ba4-d6e35aa31d50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:37.774108 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce33ab27-4c6f-4e66-8ba4-d6e35aa31d50 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:37.796025 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1417/2844] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:09:37 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:09:37.800067 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b68e77-9723-4f21-8302-0710e48fa24d admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:37.811262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b68e77-9723-4f21-8302-0710e48fa24d admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:37.816420 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f3cc5-a559-433b-b2cf-3e171a425f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:37.817682 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f3cc5-a559-433b-b2cf-3e171a425f95 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:37.817970 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f3cc5-a559-433b-b2cf-3e171a425f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:37.818359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f3cc5-a559-433b-b2cf-3e171a425f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:37.819112 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f3cc5-a559-433b-b2cf-3e171a425f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:37.820800 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f3cc5-a559-433b-b2cf-3e171a425f95 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:37.834853 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1428/2845] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:09:37.839327 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce33ab27-4c6f-4e66-8ba4-d6e35aa31d50 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:37.849869 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce33ab27-4c6f-4e66-8ba4-d6e35aa31d50 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:37.854692 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb59d46-349c-427b-8b79-9a7b76ec3729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:37.855358 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb59d46-349c-427b-8b79-9a7b76ec3729 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:37.855634 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb59d46-349c-427b-8b79-9a7b76ec3729 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:37.855901 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb59d46-349c-427b-8b79-9a7b76ec3729 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:37.856633 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb59d46-349c-427b-8b79-9a7b76ec3729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:37.858255 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb59d46-349c-427b-8b79-9a7b76ec3729 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:37.869780 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1418/2846] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:09:37.874582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f3cc5-a559-433b-b2cf-3e171a425f95 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:37.886955 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f3cc5-a559-433b-b2cf-3e171a425f95 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:37.891899 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3e08a-7eef-45ad-bb11-636f967b6796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:37.892799 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3e08a-7eef-45ad-bb11-636f967b6796 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:37.893083 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3e08a-7eef-45ad-bb11-636f967b6796 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:37.893359 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3e08a-7eef-45ad-bb11-636f967b6796 admin admin] PATH_INFO: `/v3/projects/ff8783629b6944c8ab6f93cbba5163ea/users/f6a2c1100f834ac0ad12157d48bc258b/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:37.911075 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a3e08a-7eef-45ad-bb11-636f967b6796 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff8783629b6944c8ab6f93cbba5163ea, user_id=f6a2c1100f834ac0ad12157d48bc258b, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:37.913846 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a3e08a-7eef-45ad-bb11-636f967b6796 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:37.928746 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1429/2847] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:09:37 2022] PUT /identity/v3/projects/ff8783629b6944c8ab6f93cbba5163ea/users/f6a2c1100f834ac0ad12157d48bc258b/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:09:37.933986 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b8e28-3b39-46fe-91ad-823a2fbdfdff None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:37.934299 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b8e28-3b39-46fe-91ad-823a2fbdfdff None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:37.934572 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b8e28-3b39-46fe-91ad-823a2fbdfdff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:37.957292 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-587b8e28-3b39-46fe-91ad-823a2fbdfdff None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:09:37.959130 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-587b8e28-3b39-46fe-91ad-823a2fbdfdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:38.024337 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1419/2848] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:09:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:09:38.034582 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3e08a-7eef-45ad-bb11-636f967b6796 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:38.036529 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-78a3e08a-7eef-45ad-bb11-636f967b6796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:38.096762 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3e08a-7eef-45ad-bb11-636f967b6796 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:38.100576 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdbccf-fe7d-44eb-bef8-dccccdcf78c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:38.101235 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdbccf-fe7d-44eb-bef8-dccccdcf78c2 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:38.101522 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdbccf-fe7d-44eb-bef8-dccccdcf78c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:38.101813 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdbccf-fe7d-44eb-bef8-dccccdcf78c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:38.111623 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cdbccf-fe7d-44eb-bef8-dccccdcf78c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:38.113079 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cdbccf-fe7d-44eb-bef8-dccccdcf78c2 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:38.124004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1430/2849] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:09:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:09:39.041215 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eba837-f2e9-4807-ab2f-4a0daf5fb037 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:39.041731 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eba837-f2e9-4807-ab2f-4a0daf5fb037 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:39.042055 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eba837-f2e9-4807-ab2f-4a0daf5fb037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:39.076161 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-b0eba837-f2e9-4807-ab2f-4a0daf5fb037 None None] MFA Rules not processed for user `f6a2c1100f834ac0ad12157d48bc258b`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:09:39.077409 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-b0eba837-f2e9-4807-ab2f-4a0daf5fb037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:39.108980 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:09:39.152210 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1420/2850] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:09:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:09:39.163108 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdbccf-fe7d-44eb-bef8-dccccdcf78c2 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:39.165551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e8cdbccf-fe7d-44eb-bef8-dccccdcf78c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:39.228813 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdbccf-fe7d-44eb-bef8-dccccdcf78c2 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:39.233088 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9e234-7135-4370-97a3-891398bcd26e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:39.233747 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9e234-7135-4370-97a3-891398bcd26e service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:39.234065 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9e234-7135-4370-97a3-891398bcd26e service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:39.234350 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9e234-7135-4370-97a3-891398bcd26e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:39.243346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd9e234-7135-4370-97a3-891398bcd26e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:39.244802 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd9e234-7135-4370-97a3-891398bcd26e service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:39.254214 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1431/2851] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:09:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:09:40.617995 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ac299-6dcb-4397-b9cd-5edfe2437e9e None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:40.618624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ac299-6dcb-4397-b9cd-5edfe2437e9e None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:40.618624 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ac299-6dcb-4397-b9cd-5edfe2437e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:40.638290 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-7b5ac299-6dcb-4397-b9cd-5edfe2437e9e None None] MFA Rules not processed for user `f6a2c1100f834ac0ad12157d48bc258b`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:09:40.639549 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-7b5ac299-6dcb-4397-b9cd-5edfe2437e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:40.647539 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1421/2852] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:09:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:09:49.249769 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=67,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:09:50.631868 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=17 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:09:53.663715 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9e234-7135-4370-97a3-891398bcd26e service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:53.665175 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-afd9e234-7135-4370-97a3-891398bcd26e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:53.735635 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9e234-7135-4370-97a3-891398bcd26e service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:53.741092 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f293b13-6330-44eb-871d-9cf88cc30e08 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63789c0a113447ff90933d447cf5bac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:53.741742 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f293b13-6330-44eb-871d-9cf88cc30e08 service placement] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:53.742048 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f293b13-6330-44eb-871d-9cf88cc30e08 service placement] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:53.742327 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f293b13-6330-44eb-871d-9cf88cc30e08 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:53.744096 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2f293b13-6330-44eb-871d-9cf88cc30e08 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:53.757054 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f293b13-6330-44eb-871d-9cf88cc30e08 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:53.758704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f293b13-6330-44eb-871d-9cf88cc30e08 service placement] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:53.769162 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1432/2853] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:09:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:09:54.735569 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bc4f1-3d7f-4dd4-9443-d2af2aac8efb None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:54.736218 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bc4f1-3d7f-4dd4-9443-d2af2aac8efb None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:54.736218 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bc4f1-3d7f-4dd4-9443-d2af2aac8efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:54.753467 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-9f6bc4f1-3d7f-4dd4-9443-d2af2aac8efb None None] MFA Rules not processed for user `f6a2c1100f834ac0ad12157d48bc258b`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:09:54.755202 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-9f6bc4f1-3d7f-4dd4-9443-d2af2aac8efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:54.762541 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1422/2854] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:09:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:10:03.764868 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=32 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:04.748333 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:14.994394 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d81403-b7b7-47b2-ae6f-b96cb078481e None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:14.995266 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d81403-b7b7-47b2-ae6f-b96cb078481e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:14.995463 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d81403-b7b7-47b2-ae6f-b96cb078481e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:15.057572 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-e0d81403-b7b7-47b2-ae6f-b96cb078481e None None] MFA Rules not processed for user `b2935579c8554bed8b1e211c47548ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:10:15.058872 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e0d81403-b7b7-47b2-ae6f-b96cb078481e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:15.126479 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1433/2855] 10.222.0.78 () {58 vars in 1009 bytes} [Wed May 25 15:10:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:10:15.176629 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bc4f1-3d7f-4dd4-9443-d2af2aac8efb None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:15.178365 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-9f6bc4f1-3d7f-4dd4-9443-d2af2aac8efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:15.297927 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bc4f1-3d7f-4dd4-9443-d2af2aac8efb None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:15.304447 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391636f-d967-4876-b6a9-44852a579359 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe4a2d106d84fc9ac65c046ba675353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:15.305250 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391636f-d967-4876-b6a9-44852a579359 service swift] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:15.305532 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391636f-d967-4876-b6a9-44852a579359 service swift] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:15.305807 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391636f-d967-4876-b6a9-44852a579359 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:15.314460 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391636f-d967-4876-b6a9-44852a579359 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:15.316763 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391636f-d967-4876-b6a9-44852a579359 service swift] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:15.325129 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1423/2856] 10.222.0.78 () {60 vars in 1384 bytes} [Wed May 25 15:10:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 15:10:20.555257 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d81403-b7b7-47b2-ae6f-b96cb078481e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:20.557584 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e0d81403-b7b7-47b2-ae6f-b96cb078481e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:20.573870 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d81403-b7b7-47b2-ae6f-b96cb078481e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:20.580344 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ebdc95-0a3e-42ba-8687-5602accc819d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:20.582236 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebdc95-0a3e-42ba-8687-5602accc819d service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:20.582980 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebdc95-0a3e-42ba-8687-5602accc819d service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:20.583262 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebdc95-0a3e-42ba-8687-5602accc819d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:20.585783 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e2ebdc95-0a3e-42ba-8687-5602accc819d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:20.677248 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ebdc95-0a3e-42ba-8687-5602accc819d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:20.679140 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ebdc95-0a3e-42ba-8687-5602accc819d service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:20.690860 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1434/2857] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:10:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:10:25.321769 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=41 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:27.622741 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391636f-d967-4876-b6a9-44852a579359 service swift] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:27.636728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391636f-d967-4876-b6a9-44852a579359 service swift] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:27.641575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d5374-4f88-4500-9fc9-2e46f42a2904 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:27.642438 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d5374-4f88-4500-9fc9-2e46f42a2904 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:27.642875 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d5374-4f88-4500-9fc9-2e46f42a2904 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:27.643259 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d5374-4f88-4500-9fc9-2e46f42a2904 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:27.645327 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-6a3d5374-4f88-4500-9fc9-2e46f42a2904 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:27.659132 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d5374-4f88-4500-9fc9-2e46f42a2904 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:27.660835 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d5374-4f88-4500-9fc9-2e46f42a2904 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:27.678387 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1424/2858] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:10:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:10:30.691608 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=72 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:31.475937 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5243c-aa54-4809-bf86-af44dbc5c001 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:31.476231 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5243c-aa54-4809-bf86-af44dbc5c001 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:31.476562 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5243c-aa54-4809-bf86-af44dbc5c001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:31.497326 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-9cc5243c-aa54-4809-bf86-af44dbc5c001 None None] MFA Rules not processed for user `b2935579c8554bed8b1e211c47548ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:10:31.498611 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9cc5243c-aa54-4809-bf86-af44dbc5c001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:31.506521 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1435/2859] 10.222.0.78 () {58 vars in 967 bytes} [Wed May 25 15:10:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:10:31.523316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d5374-4f88-4500-9fc9-2e46f42a2904 service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:31.537014 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d5374-4f88-4500-9fc9-2e46f42a2904 service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:31.559639 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43741e-93d1-4205-8e16-eb60929b317d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe4a2d106d84fc9ac65c046ba675353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:31.560597 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43741e-93d1-4205-8e16-eb60929b317d service swift] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:31.560873 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43741e-93d1-4205-8e16-eb60929b317d service swift] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:31.561149 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43741e-93d1-4205-8e16-eb60929b317d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:31.575417 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43741e-93d1-4205-8e16-eb60929b317d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:31.577018 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43741e-93d1-4205-8e16-eb60929b317d service swift] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:31.589815 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1425/2860] 10.222.0.78 () {60 vars in 1384 bytes} [Wed May 25 15:10:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 15:10:33.652400 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5243c-aa54-4809-bf86-af44dbc5c001 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:33.667014 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5243c-aa54-4809-bf86-af44dbc5c001 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:33.672005 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba39d7-ff55-4f98-9046-adb004755886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:33.673176 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba39d7-ff55-4f98-9046-adb004755886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:33.673550 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba39d7-ff55-4f98-9046-adb004755886 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:33.673829 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba39d7-ff55-4f98-9046-adb004755886 admin admin] PATH_INFO: `/v3/users/f6a2c1100f834ac0ad12157d48bc258b` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:33.676119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ba39d7-ff55-4f98-9046-adb004755886 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6a2c1100f834ac0ad12157d48bc258b)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:33.678527 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ba39d7-ff55-4f98-9046-adb004755886 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:33.733188 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d2ba39d7-ff55-4f98-9046-adb004755886 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6a2c1100f834ac0ad12157d48bc258b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:33.748543 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d2ba39d7-ff55-4f98-9046-adb004755886 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6a2c1100f834ac0ad12157d48bc258b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:33.757320 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d2ba39d7-ff55-4f98-9046-adb004755886 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6a2c1100f834ac0ad12157d48bc258b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:33.757993 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d2ba39d7-ff55-4f98-9046-adb004755886 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6a2c1100f834ac0ad12157d48bc258b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:33.758648 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d2ba39d7-ff55-4f98-9046-adb004755886 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6a2c1100f834ac0ad12157d48bc258b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:33.772728 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1436/2861] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:10:33 2022] DELETE /identity/v3/users/f6a2c1100f834ac0ad12157d48bc258b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:10:34.046892 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43741e-93d1-4205-8e16-eb60929b317d service swift] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:34.060456 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43741e-93d1-4205-8e16-eb60929b317d service swift] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:34.144098 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868b1de-6fb2-41bb-ae97-9a98913b33d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:34.145075 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868b1de-6fb2-41bb-ae97-9a98913b33d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:34.146556 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868b1de-6fb2-41bb-ae97-9a98913b33d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:34.147073 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868b1de-6fb2-41bb-ae97-9a98913b33d5 admin admin] PATH_INFO: `/v3/projects/ff8783629b6944c8ab6f93cbba5163ea` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:34.149848 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868b1de-6fb2-41bb-ae97-9a98913b33d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff8783629b6944c8ab6f93cbba5163ea)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:34.151912 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868b1de-6fb2-41bb-ae97-9a98913b33d5 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:34.217691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f868b1de-6fb2-41bb-ae97-9a98913b33d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff8783629b6944c8ab6f93cbba5163ea', 'request_id': 'req-f868b1de-6fb2-41bb-ae97-9a98913b33d5'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:34.231268 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f868b1de-6fb2-41bb-ae97-9a98913b33d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff8783629b6944c8ab6f93cbba5163ea', 'request_id': 'req-f868b1de-6fb2-41bb-ae97-9a98913b33d5'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:34.240430 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f868b1de-6fb2-41bb-ae97-9a98913b33d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff8783629b6944c8ab6f93cbba5163ea', 'request_id': 'req-f868b1de-6fb2-41bb-ae97-9a98913b33d5'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:34.262327 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f868b1de-6fb2-41bb-ae97-9a98913b33d5 admin admin] The token cache is being invalidate because project ff8783629b6944c8ab6f93cbba5163ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:10:34.262996 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f868b1de-6fb2-41bb-ae97-9a98913b33d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:34.263972 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f868b1de-6fb2-41bb-ae97-9a98913b33d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:34.270571 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1426/2862] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:10:34 2022] DELETE /identity/v3/projects/ff8783629b6944c8ab6f93cbba5163ea => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:10:34.272450 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddee002-8baa-4784-80f2-4234567fe2e1 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:34.273023 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddee002-8baa-4784-80f2-4234567fe2e1 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:34.273638 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddee002-8baa-4784-80f2-4234567fe2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:34.302601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-0ddee002-8baa-4784-80f2-4234567fe2e1 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:10:34.303836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-0ddee002-8baa-4784-80f2-4234567fe2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:34.395137 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1437/2863] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:10:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:10:34.403426 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d73dd-5167-4cf2-b74b-3f99df4f2602 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:34.403858 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d73dd-5167-4cf2-b74b-3f99df4f2602 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:34.404551 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d73dd-5167-4cf2-b74b-3f99df4f2602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:34.434839 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-4a3d73dd-5167-4cf2-b74b-3f99df4f2602 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:10:34.437108 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-4a3d73dd-5167-4cf2-b74b-3f99df4f2602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:34.458781 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1427/2864] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:10:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:10:34.459958 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddee002-8baa-4784-80f2-4234567fe2e1 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:34.472565 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddee002-8baa-4784-80f2-4234567fe2e1 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:34.478308 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df5708-7a4d-47f8-a1cb-4c455fa16941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:34.478991 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df5708-7a4d-47f8-a1cb-4c455fa16941 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:34.479278 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df5708-7a4d-47f8-a1cb-4c455fa16941 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:34.479557 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df5708-7a4d-47f8-a1cb-4c455fa16941 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:34.480597 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df5708-7a4d-47f8-a1cb-4c455fa16941 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:10:34.480597 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df5708-7a4d-47f8-a1cb-4c455fa16941 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:34.482418 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df5708-7a4d-47f8-a1cb-4c455fa16941 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:34.497798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1438/2865] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 15:10:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:10:34.501088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d73dd-5167-4cf2-b74b-3f99df4f2602 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:34.512048 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d73dd-5167-4cf2-b74b-3f99df4f2602 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:34.517454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f57a6a-7de4-4c02-8819-de75d93f1bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:34.518322 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f57a6a-7de4-4c02-8819-de75d93f1bc0 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:34.518726 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f57a6a-7de4-4c02-8819-de75d93f1bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:34.519113 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f57a6a-7de4-4c02-8819-de75d93f1bc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:34.519966 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f57a6a-7de4-4c02-8819-de75d93f1bc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:34.522067 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f57a6a-7de4-4c02-8819-de75d93f1bc0 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:34.556302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1428/2866] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:10:34 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:10:34.560119 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df5708-7a4d-47f8-a1cb-4c455fa16941 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:34.576511 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df5708-7a4d-47f8-a1cb-4c455fa16941 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:34.656012 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:34.669517 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800114d7-019b-492a-964c-79d46c26c2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:34.670147 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800114d7-019b-492a-964c-79d46c26c2d8 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:34.670358 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800114d7-019b-492a-964c-79d46c26c2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:34.671437 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800114d7-019b-492a-964c-79d46c26c2d8 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:34.672067 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800114d7-019b-492a-964c-79d46c26c2d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:34.674035 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800114d7-019b-492a-964c-79d46c26c2d8 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:34.705400 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1439/2867] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:10:34 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:10:34.705790 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f57a6a-7de4-4c02-8819-de75d93f1bc0 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:34.719622 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f57a6a-7de4-4c02-8819-de75d93f1bc0 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:34.725331 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac4113-4d43-4eaf-80bd-a5d9040f4710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:34.726034 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac4113-4d43-4eaf-80bd-a5d9040f4710 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:34.726328 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac4113-4d43-4eaf-80bd-a5d9040f4710 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:34.726603 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac4113-4d43-4eaf-80bd-a5d9040f4710 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:34.727334 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac4113-4d43-4eaf-80bd-a5d9040f4710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:34.729047 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac4113-4d43-4eaf-80bd-a5d9040f4710 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:34.741470 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1429/2868] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:10:34.745618 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800114d7-019b-492a-964c-79d46c26c2d8 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:34.756225 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800114d7-019b-492a-964c-79d46c26c2d8 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:34.761334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927f88f-0198-474d-a68e-f0847dc2997c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:34.762002 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927f88f-0198-474d-a68e-f0847dc2997c admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:34.762335 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927f88f-0198-474d-a68e-f0847dc2997c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:34.762613 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927f88f-0198-474d-a68e-f0847dc2997c admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:34.765278 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927f88f-0198-474d-a68e-f0847dc2997c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:34.766750 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927f88f-0198-474d-a68e-f0847dc2997c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:34.779980 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1440/2869] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:10:34.784865 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac4113-4d43-4eaf-80bd-a5d9040f4710 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:34.791894 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927f88f-0198-474d-a68e-f0847dc2997c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:34.793686 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3927f88f-0198-474d-a68e-f0847dc2997c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:34.800580 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac4113-4d43-4eaf-80bd-a5d9040f4710 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:34.808504 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927f88f-0198-474d-a68e-f0847dc2997c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:34.809715 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3c217-8c55-4c6c-abc4-b96772a6768b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:34.810654 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3c217-8c55-4c6c-abc4-b96772a6768b admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:34.811059 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3c217-8c55-4c6c-abc4-b96772a6768b admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:34.811362 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3c217-8c55-4c6c-abc4-b96772a6768b admin admin] PATH_INFO: `/v3/projects/c789ac4d5adc4658a3a26516ffc27582/users/ed0a4092f84c41f88e277a47bca330b2/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:34.814469 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf71c160-d4b9-45e2-af74-e521b962e9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:34.815357 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71c160-d4b9-45e2-af74-e521b962e9f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:34.815676 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71c160-d4b9-45e2-af74-e521b962e9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:34.815961 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71c160-d4b9-45e2-af74-e521b962e9f4 admin admin] PATH_INFO: `/v3/users/9c1bf697f0fd4e2cbc8d52cec772e446` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:34.817698 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf71c160-d4b9-45e2-af74-e521b962e9f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c1bf697f0fd4e2cbc8d52cec772e446)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:34.821335 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf71c160-d4b9-45e2-af74-e521b962e9f4 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:34.835677 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf3c217-8c55-4c6c-abc4-b96772a6768b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c789ac4d5adc4658a3a26516ffc27582, user_id=ed0a4092f84c41f88e277a47bca330b2, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:34.838675 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf3c217-8c55-4c6c-abc4-b96772a6768b admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:34.852273 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:34.860746 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1430/2870] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:10:34 2022] PUT /identity/v3/projects/c789ac4d5adc4658a3a26516ffc27582/users/ed0a4092f84c41f88e277a47bca330b2/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:10:34.880400 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e37352-1d07-4655-8f3c-7d3477720a6b None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:34.880616 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e37352-1d07-4655-8f3c-7d3477720a6b None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:34.880893 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e37352-1d07-4655-8f3c-7d3477720a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:34.887650 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-bf71c160-d4b9-45e2-af74-e521b962e9f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c1bf697f0fd4e2cbc8d52cec772e446'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:34.904674 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-bf71c160-d4b9-45e2-af74-e521b962e9f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c1bf697f0fd4e2cbc8d52cec772e446'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:34.921791 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-bf71c160-d4b9-45e2-af74-e521b962e9f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c1bf697f0fd4e2cbc8d52cec772e446'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:34.922282 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-50e37352-1d07-4655-8f3c-7d3477720a6b None None] MFA Rules not processed for user `ed0a4092f84c41f88e277a47bca330b2`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:10:34.922640 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-bf71c160-d4b9-45e2-af74-e521b962e9f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c1bf697f0fd4e2cbc8d52cec772e446'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:34.923351 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-bf71c160-d4b9-45e2-af74-e521b962e9f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c1bf697f0fd4e2cbc8d52cec772e446'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:34.924165 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-50e37352-1d07-4655-8f3c-7d3477720a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:34.943598 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1441/2871] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:10:34 2022] DELETE /identity/v3/users/9c1bf697f0fd4e2cbc8d52cec772e446 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:10:35.006372 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1431/2872] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:10:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:10:35.014141 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf71c160-d4b9-45e2-af74-e521b962e9f4 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:35.015782 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-bf71c160-d4b9-45e2-af74-e521b962e9f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:35.075511 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf71c160-d4b9-45e2-af74-e521b962e9f4 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:35.079132 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7003b-59f9-4f02-a031-63fc2313f6cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:35.079780 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7003b-59f9-4f02-a031-63fc2313f6cc service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:35.080059 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7003b-59f9-4f02-a031-63fc2313f6cc service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:35.080330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7003b-59f9-4f02-a031-63fc2313f6cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:35.126120 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e37352-1d07-4655-8f3c-7d3477720a6b None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:35.130211 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-50e37352-1d07-4655-8f3c-7d3477720a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:35.148185 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7003b-59f9-4f02-a031-63fc2313f6cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:35.149563 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7003b-59f9-4f02-a031-63fc2313f6cc service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:35.153696 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:35.165559 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1442/2873] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:10:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:10:35.224226 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e37352-1d07-4655-8f3c-7d3477720a6b None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:35.285137 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13641e4b-d9b3-455a-9646-6ee20ca7f2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:35.286030 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13641e4b-d9b3-455a-9646-6ee20ca7f2ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:35.286329 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13641e4b-d9b3-455a-9646-6ee20ca7f2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:35.286604 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13641e4b-d9b3-455a-9646-6ee20ca7f2ea admin admin] PATH_INFO: `/v3/projects/ffb2008af6964bc5b2e94a630bfe78e5` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:35.287811 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13641e4b-d9b3-455a-9646-6ee20ca7f2ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffb2008af6964bc5b2e94a630bfe78e5)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:35.289740 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13641e4b-d9b3-455a-9646-6ee20ca7f2ea admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:35.330617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:35.340841 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-13641e4b-d9b3-455a-9646-6ee20ca7f2ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffb2008af6964bc5b2e94a630bfe78e5', 'request_id': 'req-13641e4b-d9b3-455a-9646-6ee20ca7f2ea'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:35.347425 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-13641e4b-d9b3-455a-9646-6ee20ca7f2ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffb2008af6964bc5b2e94a630bfe78e5', 'request_id': 'req-13641e4b-d9b3-455a-9646-6ee20ca7f2ea'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:35.353656 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-13641e4b-d9b3-455a-9646-6ee20ca7f2ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffb2008af6964bc5b2e94a630bfe78e5', 'request_id': 'req-13641e4b-d9b3-455a-9646-6ee20ca7f2ea'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:35.366948 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-13641e4b-d9b3-455a-9646-6ee20ca7f2ea admin admin] The token cache is being invalidate because project ffb2008af6964bc5b2e94a630bfe78e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:10:35.367279 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-13641e4b-d9b3-455a-9646-6ee20ca7f2ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:35.367894 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-13641e4b-d9b3-455a-9646-6ee20ca7f2ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:35.369722 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1432/2874] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:10:35 2022] DELETE /identity/v3/projects/ffb2008af6964bc5b2e94a630bfe78e5 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:10:35.373996 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7003b-59f9-4f02-a031-63fc2313f6cc service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:35.387258 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7003b-59f9-4f02-a031-63fc2313f6cc service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:35.474259 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb6bf05-aeca-4da5-b224-ce65eda07d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:35.475148 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6bf05-aeca-4da5-b224-ce65eda07d46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:35.475432 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6bf05-aeca-4da5-b224-ce65eda07d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:35.475719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6bf05-aeca-4da5-b224-ce65eda07d46 admin admin] PATH_INFO: `/v3/users/e5081ba867b541d9ac2638a1738eb8b9` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:35.476939 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb6bf05-aeca-4da5-b224-ce65eda07d46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5081ba867b541d9ac2638a1738eb8b9)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:35.478934 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb6bf05-aeca-4da5-b224-ce65eda07d46 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:35.527218 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3eb6bf05-aeca-4da5-b224-ce65eda07d46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5081ba867b541d9ac2638a1738eb8b9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:35.539085 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3eb6bf05-aeca-4da5-b224-ce65eda07d46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5081ba867b541d9ac2638a1738eb8b9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:35.546039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3eb6bf05-aeca-4da5-b224-ce65eda07d46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5081ba867b541d9ac2638a1738eb8b9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:35.546682 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3eb6bf05-aeca-4da5-b224-ce65eda07d46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5081ba867b541d9ac2638a1738eb8b9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:35.547297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3eb6bf05-aeca-4da5-b224-ce65eda07d46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5081ba867b541d9ac2638a1738eb8b9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:35.560570 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1443/2875] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:10:35 2022] DELETE /identity/v3/users/e5081ba867b541d9ac2638a1738eb8b9 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:10:35.750185 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13641e4b-d9b3-455a-9646-6ee20ca7f2ea admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:35.751635 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-13641e4b-d9b3-455a-9646-6ee20ca7f2ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:35.765963 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13641e4b-d9b3-455a-9646-6ee20ca7f2ea admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:35.824364 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1e8db-ccc5-49ae-868e-7d4084ed7973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:35.825237 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1e8db-ccc5-49ae-868e-7d4084ed7973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:35.825532 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1e8db-ccc5-49ae-868e-7d4084ed7973 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:35.825797 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1e8db-ccc5-49ae-868e-7d4084ed7973 admin admin] PATH_INFO: `/v3/projects/11e7a4fad2d24584b2af7b50a4677ff4` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:35.827054 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1e8db-ccc5-49ae-868e-7d4084ed7973 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11e7a4fad2d24584b2af7b50a4677ff4)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:35.828981 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1e8db-ccc5-49ae-868e-7d4084ed7973 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:35.875287 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-a1f1e8db-ccc5-49ae-868e-7d4084ed7973 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11e7a4fad2d24584b2af7b50a4677ff4', 'request_id': 'req-a1f1e8db-ccc5-49ae-868e-7d4084ed7973'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:35.881330 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-a1f1e8db-ccc5-49ae-868e-7d4084ed7973 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11e7a4fad2d24584b2af7b50a4677ff4', 'request_id': 'req-a1f1e8db-ccc5-49ae-868e-7d4084ed7973'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:35.888016 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-a1f1e8db-ccc5-49ae-868e-7d4084ed7973 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11e7a4fad2d24584b2af7b50a4677ff4', 'request_id': 'req-a1f1e8db-ccc5-49ae-868e-7d4084ed7973'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:35.901071 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-a1f1e8db-ccc5-49ae-868e-7d4084ed7973 admin admin] The token cache is being invalidate because project 11e7a4fad2d24584b2af7b50a4677ff4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:10:35.901628 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-a1f1e8db-ccc5-49ae-868e-7d4084ed7973 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:35.902487 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-a1f1e8db-ccc5-49ae-868e-7d4084ed7973 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:35.904182 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1433/2876] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:10:35 2022] DELETE /identity/v3/projects/11e7a4fad2d24584b2af7b50a4677ff4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:10:37.706649 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb6bf05-aeca-4da5-b224-ce65eda07d46 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:37.708195 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3eb6bf05-aeca-4da5-b224-ce65eda07d46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:37.720946 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb6bf05-aeca-4da5-b224-ce65eda07d46 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:37.777445 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6932fc63-6f45-4ef7-900e-4cd1ca93984b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:37.778386 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932fc63-6f45-4ef7-900e-4cd1ca93984b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:37.778675 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932fc63-6f45-4ef7-900e-4cd1ca93984b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:37.778987 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932fc63-6f45-4ef7-900e-4cd1ca93984b admin admin] PATH_INFO: `/v3/users/ed0a4092f84c41f88e277a47bca330b2` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:37.780233 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6932fc63-6f45-4ef7-900e-4cd1ca93984b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed0a4092f84c41f88e277a47bca330b2)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:37.782335 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6932fc63-6f45-4ef7-900e-4cd1ca93984b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:37.804836 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:37.830679 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-6932fc63-6f45-4ef7-900e-4cd1ca93984b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed0a4092f84c41f88e277a47bca330b2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:37.843680 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-6932fc63-6f45-4ef7-900e-4cd1ca93984b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed0a4092f84c41f88e277a47bca330b2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:37.853923 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-6932fc63-6f45-4ef7-900e-4cd1ca93984b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed0a4092f84c41f88e277a47bca330b2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:37.854600 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-6932fc63-6f45-4ef7-900e-4cd1ca93984b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed0a4092f84c41f88e277a47bca330b2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:37.855223 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-6932fc63-6f45-4ef7-900e-4cd1ca93984b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed0a4092f84c41f88e277a47bca330b2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:37.868989 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1444/2877] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:10:37 2022] DELETE /identity/v3/users/ed0a4092f84c41f88e277a47bca330b2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:10:37.873497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb8de2-1947-437b-b560-f2d4e17432af None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:37.873992 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb8de2-1947-437b-b560-f2d4e17432af None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:37.874138 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb8de2-1947-437b-b560-f2d4e17432af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:37.895096 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-6feb8de2-1947-437b-b560-f2d4e17432af None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:10:37.896513 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-6feb8de2-1947-437b-b560-f2d4e17432af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:37.948656 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:37.960247 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1434/2878] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:10:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:10:37.969701 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6932fc63-6f45-4ef7-900e-4cd1ca93984b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:37.971153 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-6932fc63-6f45-4ef7-900e-4cd1ca93984b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:38.035665 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6932fc63-6f45-4ef7-900e-4cd1ca93984b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:38.039257 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d9c12-f74e-4d34-9c37-93d0cf397d83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:38.039917 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d9c12-f74e-4d34-9c37-93d0cf397d83 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:38.040198 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d9c12-f74e-4d34-9c37-93d0cf397d83 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:38.040474 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d9c12-f74e-4d34-9c37-93d0cf397d83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:38.050711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d9c12-f74e-4d34-9c37-93d0cf397d83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:38.052189 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d9c12-f74e-4d34-9c37-93d0cf397d83 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:38.063045 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1445/2879] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:10:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:10:38.346401 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb8de2-1947-437b-b560-f2d4e17432af None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:38.347899 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-6feb8de2-1947-437b-b560-f2d4e17432af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:38.360178 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb8de2-1947-437b-b560-f2d4e17432af None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:38.364783 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6328341-cb32-491c-a054-97ca7eae49ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:38.365654 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6328341-cb32-491c-a054-97ca7eae49ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:38.365935 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6328341-cb32-491c-a054-97ca7eae49ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:38.366243 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6328341-cb32-491c-a054-97ca7eae49ca admin admin] PATH_INFO: `/v3/projects/c789ac4d5adc4658a3a26516ffc27582` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:38.367497 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6328341-cb32-491c-a054-97ca7eae49ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=c789ac4d5adc4658a3a26516ffc27582)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:38.369481 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6328341-cb32-491c-a054-97ca7eae49ca admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:38.422219 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f6328341-cb32-491c-a054-97ca7eae49ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c789ac4d5adc4658a3a26516ffc27582', 'request_id': 'req-f6328341-cb32-491c-a054-97ca7eae49ca'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:38.427890 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f6328341-cb32-491c-a054-97ca7eae49ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c789ac4d5adc4658a3a26516ffc27582', 'request_id': 'req-f6328341-cb32-491c-a054-97ca7eae49ca'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:38.434503 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f6328341-cb32-491c-a054-97ca7eae49ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c789ac4d5adc4658a3a26516ffc27582', 'request_id': 'req-f6328341-cb32-491c-a054-97ca7eae49ca'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:38.450575 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f6328341-cb32-491c-a054-97ca7eae49ca admin admin] The token cache is being invalidate because project c789ac4d5adc4658a3a26516ffc27582 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86692) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:10:38.450891 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f6328341-cb32-491c-a054-97ca7eae49ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:38.451558 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-f6328341-cb32-491c-a054-97ca7eae49ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:10:38.453180 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1435/2880] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:10:38 2022] DELETE /identity/v3/projects/c789ac4d5adc4658a3a26516ffc27582 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:10:38.459032 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853f788-f8bf-4f59-a10d-c5f0c0a7f55b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:38.459326 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853f788-f8bf-4f59-a10d-c5f0c0a7f55b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:38.459605 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853f788-f8bf-4f59-a10d-c5f0c0a7f55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:38.477487 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-8853f788-f8bf-4f59-a10d-c5f0c0a7f55b None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:10:38.478838 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8853f788-f8bf-4f59-a10d-c5f0c0a7f55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:38.542600 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1446/2881] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:10:38.551100 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be01187-f3e6-4710-a855-6b7ca41c3fdf None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:38.551385 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be01187-f3e6-4710-a855-6b7ca41c3fdf None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:38.551665 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be01187-f3e6-4710-a855-6b7ca41c3fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:38.570625 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-2be01187-f3e6-4710-a855-6b7ca41c3fdf None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:10:38.571835 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-2be01187-f3e6-4710-a855-6b7ca41c3fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:38.579872 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1436/2882] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:10:38.585995 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853f788-f8bf-4f59-a10d-c5f0c0a7f55b None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:38.595890 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853f788-f8bf-4f59-a10d-c5f0c0a7f55b None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:38.600644 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b08d1a-899e-4ce4-bf58-6ccc896c96e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:38.601420 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b08d1a-899e-4ce4-bf58-6ccc896c96e4 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:38.602036 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b08d1a-899e-4ce4-bf58-6ccc896c96e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:38.602337 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b08d1a-899e-4ce4-bf58-6ccc896c96e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:38.603055 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b08d1a-899e-4ce4-bf58-6ccc896c96e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:10:38.603360 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b08d1a-899e-4ce4-bf58-6ccc896c96e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:38.605005 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b08d1a-899e-4ce4-bf58-6ccc896c96e4 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:38.618068 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1447/2883] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 15:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:10:38.622718 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be01187-f3e6-4710-a855-6b7ca41c3fdf None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:38.632904 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be01187-f3e6-4710-a855-6b7ca41c3fdf None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:38.637946 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd00ca6-aa42-4091-9501-9aca8578e7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:38.638634 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd00ca6-aa42-4091-9501-9aca8578e7cf admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:38.638917 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd00ca6-aa42-4091-9501-9aca8578e7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:38.639191 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd00ca6-aa42-4091-9501-9aca8578e7cf admin admin] PATH_INFO: `/v3/projects` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:38.639880 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd00ca6-aa42-4091-9501-9aca8578e7cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:38.641783 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd00ca6-aa42-4091-9501-9aca8578e7cf admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:38.668819 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1437/2884] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:10:38 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:10:38.673442 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b08d1a-899e-4ce4-bf58-6ccc896c96e4 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:38.684125 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b08d1a-899e-4ce4-bf58-6ccc896c96e4 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:38.709016 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:38.749437 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5addc442-5814-4ad0-99bb-448101463116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:38.750281 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5addc442-5814-4ad0-99bb-448101463116 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:38.750572 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5addc442-5814-4ad0-99bb-448101463116 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:38.750847 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5addc442-5814-4ad0-99bb-448101463116 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:38.751577 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5addc442-5814-4ad0-99bb-448101463116 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:38.755012 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5addc442-5814-4ad0-99bb-448101463116 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:38.776418 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1448/2885] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:10:38 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:10:38.780728 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd00ca6-aa42-4091-9501-9aca8578e7cf admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:38.791454 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd00ca6-aa42-4091-9501-9aca8578e7cf admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:38.796384 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5dc972-6f84-489b-b3cf-ca7d17b74ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:38.796977 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5dc972-6f84-489b-b3cf-ca7d17b74ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:38.797283 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5dc972-6f84-489b-b3cf-ca7d17b74ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:38.797536 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5dc972-6f84-489b-b3cf-ca7d17b74ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:38.798290 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5dc972-6f84-489b-b3cf-ca7d17b74ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:38.800552 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5dc972-6f84-489b-b3cf-ca7d17b74ec3 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:38.811660 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1438/2886] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:10:38.816412 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5addc442-5814-4ad0-99bb-448101463116 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:38.826559 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5addc442-5814-4ad0-99bb-448101463116 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:38.831277 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55533e38-2af8-42a6-8293-48744156a077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:38.831940 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55533e38-2af8-42a6-8293-48744156a077 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:38.832230 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55533e38-2af8-42a6-8293-48744156a077 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:38.832513 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55533e38-2af8-42a6-8293-48744156a077 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:38.833274 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55533e38-2af8-42a6-8293-48744156a077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:38.834936 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55533e38-2af8-42a6-8293-48744156a077 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:38.847048 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1449/2887] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:10:38.851530 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5dc972-6f84-489b-b3cf-ca7d17b74ec3 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:38.861783 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5dc972-6f84-489b-b3cf-ca7d17b74ec3 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:38.867150 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a99652-ca21-44ad-b266-4e34ec71faa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:38.867973 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a99652-ca21-44ad-b266-4e34ec71faa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:38.868256 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a99652-ca21-44ad-b266-4e34ec71faa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:38.868632 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a99652-ca21-44ad-b266-4e34ec71faa8 admin admin] PATH_INFO: `/v3/projects/19123d88f28441d1a7bf8ea3e7227de2/users/cc9d991e73a84f3599af9269efed898d/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:38.884018 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a99652-ca21-44ad-b266-4e34ec71faa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19123d88f28441d1a7bf8ea3e7227de2, user_id=cc9d991e73a84f3599af9269efed898d, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:38.887022 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a99652-ca21-44ad-b266-4e34ec71faa8 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:38.900687 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1439/2888] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:10:38 2022] PUT /identity/v3/projects/19123d88f28441d1a7bf8ea3e7227de2/users/cc9d991e73a84f3599af9269efed898d/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:10:38.905977 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e6550-742a-44ff-9893-1d8c1e3e004d None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:38.906145 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e6550-742a-44ff-9893-1d8c1e3e004d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:38.906401 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e6550-742a-44ff-9893-1d8c1e3e004d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:38.923820 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-972e6550-742a-44ff-9893-1d8c1e3e004d None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:10:38.925545 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-972e6550-742a-44ff-9893-1d8c1e3e004d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:38.987982 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1450/2889] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:10:38.997967 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a99652-ca21-44ad-b266-4e34ec71faa8 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:38.999720 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-d7a99652-ca21-44ad-b266-4e34ec71faa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:39.064502 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a99652-ca21-44ad-b266-4e34ec71faa8 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:39.068310 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4558a48-ffe0-4e70-8a1d-8d0c38b05eb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:39.068952 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4558a48-ffe0-4e70-8a1d-8d0c38b05eb5 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:39.069228 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4558a48-ffe0-4e70-8a1d-8d0c38b05eb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:39.069502 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4558a48-ffe0-4e70-8a1d-8d0c38b05eb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:39.079172 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4558a48-ffe0-4e70-8a1d-8d0c38b05eb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:39.080617 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4558a48-ffe0-4e70-8a1d-8d0c38b05eb5 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:39.090948 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1440/2890] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:10:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:10:39.735523 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809074c2-f115-4fa8-a660-f7f55d261441 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:39.736072 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809074c2-f115-4fa8-a660-f7f55d261441 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:39.736566 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809074c2-f115-4fa8-a660-f7f55d261441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:39.771448 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-809074c2-f115-4fa8-a660-f7f55d261441 None None] MFA Rules not processed for user `cc9d991e73a84f3599af9269efed898d`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:10:39.772702 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-809074c2-f115-4fa8-a660-f7f55d261441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:39.847684 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1451/2891] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:10:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:10:39.859564 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4558a48-ffe0-4e70-8a1d-8d0c38b05eb5 service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:39.861630 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-e4558a48-ffe0-4e70-8a1d-8d0c38b05eb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:39.898545 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:39.944780 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4558a48-ffe0-4e70-8a1d-8d0c38b05eb5 service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:39.948484 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b759be-5fc8-40ad-ab1f-aa2d55ab89de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:39.949138 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b759be-5fc8-40ad-ab1f-aa2d55ab89de service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:39.949413 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b759be-5fc8-40ad-ab1f-aa2d55ab89de service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:39.949689 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b759be-5fc8-40ad-ab1f-aa2d55ab89de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:39.959685 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b759be-5fc8-40ad-ab1f-aa2d55ab89de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:39.961211 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b759be-5fc8-40ad-ab1f-aa2d55ab89de service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:39.971316 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1441/2892] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:10:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:10:41.404687 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f7d92-4838-4344-8953-b55dce127192 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:41.405261 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f7d92-4838-4344-8953-b55dce127192 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:41.405261 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f7d92-4838-4344-8953-b55dce127192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:41.424297 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-480f7d92-4838-4344-8953-b55dce127192 None None] MFA Rules not processed for user `cc9d991e73a84f3599af9269efed898d`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:10:41.426514 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-480f7d92-4838-4344-8953-b55dce127192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:41.433711 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1452/2893] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:10:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:10:41.697213 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b759be-5fc8-40ad-ab1f-aa2d55ab89de service cinder] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:41.698886 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-b5b759be-5fc8-40ad-ab1f-aa2d55ab89de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:41.713090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b759be-5fc8-40ad-ab1f-aa2d55ab89de service cinder] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:41.717096 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9056938-bc3a-4ae9-82d7-34a35f46bec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:41.717773 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9056938-bc3a-4ae9-82d7-34a35f46bec3 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:41.718079 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9056938-bc3a-4ae9-82d7-34a35f46bec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:41.718358 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9056938-bc3a-4ae9-82d7-34a35f46bec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:41.720411 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-e9056938-bc3a-4ae9-82d7-34a35f46bec3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:41.732288 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9056938-bc3a-4ae9-82d7-34a35f46bec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:41.733750 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9056938-bc3a-4ae9-82d7-34a35f46bec3 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:41.742990 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1442/2894] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:10:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:10:51.418347 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:51.738962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=23 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:11:08.018566 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f7d92-4838-4344-8953-b55dce127192 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:11:08.020627 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-480f7d92-4838-4344-8953-b55dce127192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:11:08.033485 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f7d92-4838-4344-8953-b55dce127192 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:11:08.037823 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf25812-d2d9-4ce2-be7b-e88f2c30e73a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:11:08.038534 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf25812-d2d9-4ce2-be7b-e88f2c30e73a service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:11:08.038821 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf25812-d2d9-4ce2-be7b-e88f2c30e73a service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:11:08.039100 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf25812-d2d9-4ce2-be7b-e88f2c30e73a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:11:08.041063 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2cf25812-d2d9-4ce2-be7b-e88f2c30e73a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:11:08.120940 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf25812-d2d9-4ce2-be7b-e88f2c30e73a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:11:08.122460 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf25812-d2d9-4ce2-be7b-e88f2c30e73a service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:11:08.136526 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1453/2895] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:11:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:11:18.129832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=34 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:11:31.185171 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9056938-bc3a-4ae9-82d7-34a35f46bec3 service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:11:31.196744 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9056938-bc3a-4ae9-82d7-34a35f46bec3 service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:11:31.200431 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1600b19-9469-4399-8d55-6b2f9a6c14bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:11:31.201103 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1600b19-9469-4399-8d55-6b2f9a6c14bb service cinder] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:11:31.201388 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1600b19-9469-4399-8d55-6b2f9a6c14bb service cinder] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:11:31.201663 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1600b19-9469-4399-8d55-6b2f9a6c14bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:11:31.203546 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-f1600b19-9469-4399-8d55-6b2f9a6c14bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:11:31.220868 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1600b19-9469-4399-8d55-6b2f9a6c14bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:11:31.222353 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1600b19-9469-4399-8d55-6b2f9a6c14bb service cinder] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:11:31.238317 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1443/2896] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:11:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:11:41.229512 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=6 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:11:44.002392 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fee166-fd3c-4adf-b8f1-d2d52bb00d6d None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:11:44.002907 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fee166-fd3c-4adf-b8f1-d2d52bb00d6d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:11:44.003017 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fee166-fd3c-4adf-b8f1-d2d52bb00d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:11:44.020623 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-78fee166-fd3c-4adf-b8f1-d2d52bb00d6d None None] MFA Rules not processed for user `cc9d991e73a84f3599af9269efed898d`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:11:44.021831 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-78fee166-fd3c-4adf-b8f1-d2d52bb00d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:11:44.029092 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1454/2897] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:11:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:11:44.258059 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1600b19-9469-4399-8d55-6b2f9a6c14bb service cinder] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:11:44.267541 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1600b19-9469-4399-8d55-6b2f9a6c14bb service cinder] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:11:44.271133 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26532a-6be5-4e2a-aec4-ae06079947b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:11:44.271757 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26532a-6be5-4e2a-aec4-ae06079947b5 service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:11:44.272035 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26532a-6be5-4e2a-aec4-ae06079947b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:11:44.272302 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26532a-6be5-4e2a-aec4-ae06079947b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:11:44.274374 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-6e26532a-6be5-4e2a-aec4-ae06079947b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:11:44.285360 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26532a-6be5-4e2a-aec4-ae06079947b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:11:44.286864 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26532a-6be5-4e2a-aec4-ae06079947b5 service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:11:44.296092 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1444/2898] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:11:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:11:44.608641 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fee166-fd3c-4adf-b8f1-d2d52bb00d6d None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:11:44.610350 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-78fee166-fd3c-4adf-b8f1-d2d52bb00d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:11:44.695970 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fee166-fd3c-4adf-b8f1-d2d52bb00d6d None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:11:44.700973 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44a1da-2404-4133-be0b-ac8ab547c4da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63789c0a113447ff90933d447cf5bac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:11:44.701624 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44a1da-2404-4133-be0b-ac8ab547c4da service placement] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:11:44.701912 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44a1da-2404-4133-be0b-ac8ab547c4da service placement] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:11:44.702220 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44a1da-2404-4133-be0b-ac8ab547c4da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:11:44.704018 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9b44a1da-2404-4133-be0b-ac8ab547c4da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:11:44.716371 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b44a1da-2404-4133-be0b-ac8ab547c4da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:11:44.718058 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b44a1da-2404-4133-be0b-ac8ab547c4da service placement] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:11:44.728798 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1455/2899] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:11:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:11:46.452998 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26532a-6be5-4e2a-aec4-ae06079947b5 service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:11:46.463304 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26532a-6be5-4e2a-aec4-ae06079947b5 service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:11:46.467372 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ffb975-cc52-4905-8e43-98d1b7197d8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:11:46.468027 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffb975-cc52-4905-8e43-98d1b7197d8b service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:11:46.468312 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffb975-cc52-4905-8e43-98d1b7197d8b service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:11:46.468590 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffb975-cc52-4905-8e43-98d1b7197d8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:11:46.470474 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-63ffb975-cc52-4905-8e43-98d1b7197d8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:11:46.481064 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ffb975-cc52-4905-8e43-98d1b7197d8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:11:46.482551 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ffb975-cc52-4905-8e43-98d1b7197d8b service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:11:46.491335 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1445/2900] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:11:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:11:47.178891 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44a1da-2404-4133-be0b-ac8ab547c4da service placement] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:11:47.181435 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9b44a1da-2404-4133-be0b-ac8ab547c4da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:11:47.198587 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44a1da-2404-4133-be0b-ac8ab547c4da service placement] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:11:47.204346 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250af991-29e8-471e-a340-af98690dda80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e34e1d7ffbac49b0bde45829af0c6925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:11:47.205004 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250af991-29e8-471e-a340-af98690dda80 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:11:47.205293 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250af991-29e8-471e-a340-af98690dda80 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:11:47.205573 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250af991-29e8-471e-a340-af98690dda80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:11:47.207422 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-250af991-29e8-471e-a340-af98690dda80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:11:47.220424 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250af991-29e8-471e-a340-af98690dda80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:11:47.221942 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250af991-29e8-471e-a340-af98690dda80 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:11:47.233508 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1456/2901] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:11:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:11:56.487332 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=12 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:11:57.228923 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=41 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:12:41.319652 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ffb975-cc52-4905-8e43-98d1b7197d8b service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:12:41.332567 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ffb975-cc52-4905-8e43-98d1b7197d8b service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:12:41.339333 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1f0af-2e8a-4acc-80d8-40df05be14c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:12:41.340267 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1f0af-2e8a-4acc-80d8-40df05be14c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:12:41.340538 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1f0af-2e8a-4acc-80d8-40df05be14c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:12:41.340817 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1f0af-2e8a-4acc-80d8-40df05be14c1 admin admin] PATH_INFO: `/v3/users/cc9d991e73a84f3599af9269efed898d` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:12:41.342096 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1f0af-2e8a-4acc-80d8-40df05be14c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc9d991e73a84f3599af9269efed898d)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:12:41.344144 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1f0af-2e8a-4acc-80d8-40df05be14c1 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:12:41.404400 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-42b1f0af-2e8a-4acc-80d8-40df05be14c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc9d991e73a84f3599af9269efed898d'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:12:41.419217 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-42b1f0af-2e8a-4acc-80d8-40df05be14c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc9d991e73a84f3599af9269efed898d'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:12:41.429478 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-42b1f0af-2e8a-4acc-80d8-40df05be14c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc9d991e73a84f3599af9269efed898d'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:12:41.430144 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-42b1f0af-2e8a-4acc-80d8-40df05be14c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc9d991e73a84f3599af9269efed898d'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:12:41.430771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-42b1f0af-2e8a-4acc-80d8-40df05be14c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc9d991e73a84f3599af9269efed898d'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:12:41.449352 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1446/2902] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:12:41 2022] DELETE /identity/v3/users/cc9d991e73a84f3599af9269efed898d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:12:41.626950 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250af991-29e8-471e-a340-af98690dda80 service nova] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:12:41.648306 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250af991-29e8-471e-a340-af98690dda80 service nova] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:12:41.706550 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d9759-1bc7-44aa-9d10-788c0c67a184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:12:41.707334 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d9759-1bc7-44aa-9d10-788c0c67a184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:12:41.707616 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d9759-1bc7-44aa-9d10-788c0c67a184 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:12:41.707891 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d9759-1bc7-44aa-9d10-788c0c67a184 admin admin] PATH_INFO: `/v3/projects/19123d88f28441d1a7bf8ea3e7227de2` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:12:41.709062 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364d9759-1bc7-44aa-9d10-788c0c67a184 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19123d88f28441d1a7bf8ea3e7227de2)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:12:41.711005 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364d9759-1bc7-44aa-9d10-788c0c67a184 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:12:41.758776 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-364d9759-1bc7-44aa-9d10-788c0c67a184 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19123d88f28441d1a7bf8ea3e7227de2', 'request_id': 'req-364d9759-1bc7-44aa-9d10-788c0c67a184'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:12:41.764472 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-364d9759-1bc7-44aa-9d10-788c0c67a184 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19123d88f28441d1a7bf8ea3e7227de2', 'request_id': 'req-364d9759-1bc7-44aa-9d10-788c0c67a184'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:12:41.770774 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-364d9759-1bc7-44aa-9d10-788c0c67a184 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19123d88f28441d1a7bf8ea3e7227de2', 'request_id': 'req-364d9759-1bc7-44aa-9d10-788c0c67a184'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:12:41.784069 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-364d9759-1bc7-44aa-9d10-788c0c67a184 admin admin] The token cache is being invalidate because project 19123d88f28441d1a7bf8ea3e7227de2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:12:41.784358 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-364d9759-1bc7-44aa-9d10-788c0c67a184 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:12:41.785001 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-364d9759-1bc7-44aa-9d10-788c0c67a184 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:12:41.786601 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1457/2903] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:12:41 2022] DELETE /identity/v3/projects/19123d88f28441d1a7bf8ea3e7227de2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:12:41.791861 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f8e17-e5e5-4495-8d18-532d80cc3d77 None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:12:41.792150 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f8e17-e5e5-4495-8d18-532d80cc3d77 None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:12:41.792424 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f8e17-e5e5-4495-8d18-532d80cc3d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:12:41.811709 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-a48f8e17-e5e5-4495-8d18-532d80cc3d77 None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:12:41.812923 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-a48f8e17-e5e5-4495-8d18-532d80cc3d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:12:41.871692 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1447/2904] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:12:41.879220 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133e77f-e6d0-44ce-a83e-a38fbfbbaadf None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:12:41.879513 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133e77f-e6d0-44ce-a83e-a38fbfbbaadf None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:12:41.879797 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133e77f-e6d0-44ce-a83e-a38fbfbbaadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:12:41.899237 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-d133e77f-e6d0-44ce-a83e-a38fbfbbaadf None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:12:41.900704 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d133e77f-e6d0-44ce-a83e-a38fbfbbaadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:12:41.909517 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1458/2905] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:12:41.914415 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f8e17-e5e5-4495-8d18-532d80cc3d77 None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:12:41.924155 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f8e17-e5e5-4495-8d18-532d80cc3d77 None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:12:41.928723 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc036c8-8d94-4658-9029-ab7a95b2e0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:12:41.929376 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc036c8-8d94-4658-9029-ab7a95b2e0a0 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:12:41.929653 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc036c8-8d94-4658-9029-ab7a95b2e0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:12:41.929924 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc036c8-8d94-4658-9029-ab7a95b2e0a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:12:41.930678 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc036c8-8d94-4658-9029-ab7a95b2e0a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86692) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:12:41.930975 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc036c8-8d94-4658-9029-ab7a95b2e0a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:12:41.932559 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc036c8-8d94-4658-9029-ab7a95b2e0a0 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:12:41.944438 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1448/2906] 10.222.0.78 () {58 vars in 1152 bytes} [Wed May 25 15:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:12:41.948839 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133e77f-e6d0-44ce-a83e-a38fbfbbaadf None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:12:41.958248 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133e77f-e6d0-44ce-a83e-a38fbfbbaadf None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:12:41.963284 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d8eb7-47c3-48fa-86e3-c62db1d777d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:12:41.963926 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d8eb7-47c3-48fa-86e3-c62db1d777d6 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:12:41.964212 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d8eb7-47c3-48fa-86e3-c62db1d777d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:12:41.964479 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d8eb7-47c3-48fa-86e3-c62db1d777d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:12:41.965168 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d8eb7-47c3-48fa-86e3-c62db1d777d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:12:41.967113 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d8eb7-47c3-48fa-86e3-c62db1d777d6 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:12:41.992764 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1459/2907] 10.222.0.78 () {60 vars in 1152 bytes} [Wed May 25 15:12:41 2022] POST /identity/v3/projects => generated 391 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:12:41.996588 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc036c8-8d94-4658-9029-ab7a95b2e0a0 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:12:42.006529 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc036c8-8d94-4658-9029-ab7a95b2e0a0 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:12:42.065142 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b4179-1289-49e4-a9b6-a7f5fa889243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:12:42.065815 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b4179-1289-49e4-a9b6-a7f5fa889243 admin admin] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:12:42.066123 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b4179-1289-49e4-a9b6-a7f5fa889243 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:12:42.066396 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b4179-1289-49e4-a9b6-a7f5fa889243 admin admin] PATH_INFO: `/v3/users` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:12:42.067088 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b4179-1289-49e4-a9b6-a7f5fa889243 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:12:42.068981 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b4179-1289-49e4-a9b6-a7f5fa889243 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:12:42.075669 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:12:42.091017 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1449/2908] 10.222.0.78 () {60 vars in 1143 bytes} [Wed May 25 15:12:41 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:12:42.095378 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d8eb7-47c3-48fa-86e3-c62db1d777d6 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:12:42.106575 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d8eb7-47c3-48fa-86e3-c62db1d777d6 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:12:42.111246 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6ec46-54f2-4f87-9c19-65b88731e246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:12:42.111902 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6ec46-54f2-4f87-9c19-65b88731e246 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:12:42.112183 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6ec46-54f2-4f87-9c19-65b88731e246 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:12:42.112456 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6ec46-54f2-4f87-9c19-65b88731e246 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:12:42.113172 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d6ec46-54f2-4f87-9c19-65b88731e246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:12:42.114826 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d6ec46-54f2-4f87-9c19-65b88731e246 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:12:42.125325 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1460/2909] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:12:42.129771 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b4179-1289-49e4-a9b6-a7f5fa889243 admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:12:42.144801 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b4179-1289-49e4-a9b6-a7f5fa889243 admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:12:42.150090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8968699b-5ee4-4b3d-94e3-35eaf01e9d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:12:42.150864 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968699b-5ee4-4b3d-94e3-35eaf01e9d27 admin admin] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:12:42.151147 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968699b-5ee4-4b3d-94e3-35eaf01e9d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:12:42.151423 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968699b-5ee4-4b3d-94e3-35eaf01e9d27 admin admin] PATH_INFO: `/v3/roles` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:12:42.152146 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8968699b-5ee4-4b3d-94e3-35eaf01e9d27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:12:42.153734 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8968699b-5ee4-4b3d-94e3-35eaf01e9d27 admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:12:42.165168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1450/2910] 10.222.0.78 () {58 vars in 1121 bytes} [Wed May 25 15:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:12:42.169406 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6ec46-54f2-4f87-9c19-65b88731e246 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:12:42.179716 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6ec46-54f2-4f87-9c19-65b88731e246 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:12:42.186673 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb9d91-03a7-4a3d-909a-6eea65b29dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:12:42.187986 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb9d91-03a7-4a3d-909a-6eea65b29dc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:12:42.188311 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb9d91-03a7-4a3d-909a-6eea65b29dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:12:42.188681 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb9d91-03a7-4a3d-909a-6eea65b29dc5 admin admin] PATH_INFO: `/v3/projects/27d90d33b7f84b16b962b6bddba72047/users/9e9532d3e2d945eab02fbf7b6a8f667b/roles/a3da3c62738d4790b9dac512b1568b52` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:12:42.206396 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb9d91-03a7-4a3d-909a-6eea65b29dc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27d90d33b7f84b16b962b6bddba72047, user_id=9e9532d3e2d945eab02fbf7b6a8f667b, role_id=a3da3c62738d4790b9dac512b1568b52)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:12:42.209719 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb9d91-03a7-4a3d-909a-6eea65b29dc5 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:12:42.223908 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1461/2911] 10.222.0.78 () {60 vars in 1482 bytes} [Wed May 25 15:12:42 2022] PUT /identity/v3/projects/27d90d33b7f84b16b962b6bddba72047/users/9e9532d3e2d945eab02fbf7b6a8f667b/roles/a3da3c62738d4790b9dac512b1568b52 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:12:42.235082 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7dd70c-6c3c-4030-8cc2-82bac2d177de None None] REQUEST_METHOD: `POST` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:12:42.235365 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7dd70c-6c3c-4030-8cc2-82bac2d177de None None] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:12:42.235641 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7dd70c-6c3c-4030-8cc2-82bac2d177de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:12:42.263950 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.auth.core [None req-2a7dd70c-6c3c-4030-8cc2-82bac2d177de None None] MFA Rules not processed for user `9e9532d3e2d945eab02fbf7b6a8f667b`. Rule list: `[]` (Enabled: `True`). {{(pid=86692) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:12:42.265168 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-2a7dd70c-6c3c-4030-8cc2-82bac2d177de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:12:42.331698 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1451/2912] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:12:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:12:42.343071 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb9d91-03a7-4a3d-909a-6eea65b29dc5 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:12:42.344754 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ebdb9d91-03a7-4a3d-909a-6eea65b29dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:12:42.377964 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:12:42.410160 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb9d91-03a7-4a3d-909a-6eea65b29dc5 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:12:42.416895 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7cea60-0b99-4f2c-9ad7-13a834000d9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50dbd3dd7e87481cb940a467c294006c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:12:42.417570 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cea60-0b99-4f2c-9ad7-13a834000d9d service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:12:42.417854 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cea60-0b99-4f2c-9ad7-13a834000d9d service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:12:42.418166 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cea60-0b99-4f2c-9ad7-13a834000d9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:12:42.427463 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7cea60-0b99-4f2c-9ad7-13a834000d9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:12:42.428939 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7cea60-0b99-4f2c-9ad7-13a834000d9d service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:12:42.438787 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1462/2913] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:12:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:12:52.328672 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:12:52.434316 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=14 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:12:58.621300 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7dd70c-6c3c-4030-8cc2-82bac2d177de None None] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:12:58.716708 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7dd70c-6c3c-4030-8cc2-82bac2d177de None None] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:12:58.722573 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd6cea-0dbf-412c-b719-5fa2729b45de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:12:58.723583 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd6cea-0dbf-412c-b719-5fa2729b45de admin admin] REQUEST_METHOD: `DELETE` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:12:58.724005 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd6cea-0dbf-412c-b719-5fa2729b45de admin admin] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:12:58.724442 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd6cea-0dbf-412c-b719-5fa2729b45de admin admin] PATH_INFO: `/v3/users/9e9532d3e2d945eab02fbf7b6a8f667b` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:12:58.725897 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd6cea-0dbf-412c-b719-5fa2729b45de admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e9532d3e2d945eab02fbf7b6a8f667b)` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:12:58.728794 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd6cea-0dbf-412c-b719-5fa2729b45de admin admin] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:12:58.785477 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-49fd6cea-0dbf-412c-b719-5fa2729b45de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e9532d3e2d945eab02fbf7b6a8f667b'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:12:58.797761 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-49fd6cea-0dbf-412c-b719-5fa2729b45de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e9532d3e2d945eab02fbf7b6a8f667b'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:12:58.806115 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-49fd6cea-0dbf-412c-b719-5fa2729b45de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e9532d3e2d945eab02fbf7b6a8f667b'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:12:58.806800 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-49fd6cea-0dbf-412c-b719-5fa2729b45de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e9532d3e2d945eab02fbf7b6a8f667b'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:12:58.807403 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.notifications [None req-49fd6cea-0dbf-412c-b719-5fa2729b45de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e9532d3e2d945eab02fbf7b6a8f667b'} {{(pid=86692) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:12:58.820962 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1452/2914] 10.222.0.78 () {58 vars in 1223 bytes} [Wed May 25 15:12:58 2022] DELETE /identity/v3/users/9e9532d3e2d945eab02fbf7b6a8f667b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:12:58.825484 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c9e63-3d14-4b13-85bf-734d46a776aa None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:12:58.825998 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c9e63-3d14-4b13-85bf-734d46a776aa None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:12:58.826141 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c9e63-3d14-4b13-85bf-734d46a776aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:12:58.844330 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-dc8c9e63-3d14-4b13-85bf-734d46a776aa None None] MFA Rules not processed for user `f27fbbbc33b24630948a570ccae5fad0`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:12:58.845570 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-dc8c9e63-3d14-4b13-85bf-734d46a776aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:12:58.908644 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1463/2915] 10.222.0.78 () {58 vars in 957 bytes} [Wed May 25 15:12:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:12:58.918550 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd6cea-0dbf-412c-b719-5fa2729b45de admin admin] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:12:58.920099 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-49fd6cea-0dbf-412c-b719-5fa2729b45de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:12:58.977701 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd6cea-0dbf-412c-b719-5fa2729b45de admin admin] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:12:58.981445 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af273b41-36e5-4c09-8eb2-625ea6c3c57d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70ad51a8ad68440da6618891e2e08cb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:12:58.982172 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af273b41-36e5-4c09-8eb2-625ea6c3c57d service neutron] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:12:58.982444 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af273b41-36e5-4c09-8eb2-625ea6c3c57d service neutron] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:12:58.982717 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af273b41-36e5-4c09-8eb2-625ea6c3c57d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:12:58.992357 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af273b41-36e5-4c09-8eb2-625ea6c3c57d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:12:58.993820 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af273b41-36e5-4c09-8eb2-625ea6c3c57d service neutron] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:12:59.004457 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1453/2916] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:12:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:12:59.304684 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c9e63-3d14-4b13-85bf-734d46a776aa None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:12:59.306199 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-dc8c9e63-3d14-4b13-85bf-734d46a776aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:12:59.319133 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c9e63-3d14-4b13-85bf-734d46a776aa None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:12:59.323887 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948d726-1568-4399-a8f8-44676babe1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27fbbbc33b24630948a570ccae5fad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86b029bd8d8948dfab182a990de56b86', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:12:59.324755 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948d726-1568-4399-a8f8-44676babe1a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:12:59.325039 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948d726-1568-4399-a8f8-44676babe1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:12:59.325312 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948d726-1568-4399-a8f8-44676babe1a5 admin admin] PATH_INFO: `/v3/projects/27d90d33b7f84b16b962b6bddba72047` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:12:59.326551 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5948d726-1568-4399-a8f8-44676babe1a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27d90d33b7f84b16b962b6bddba72047)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:12:59.328544 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5948d726-1568-4399-a8f8-44676babe1a5 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:12:59.379102 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5948d726-1568-4399-a8f8-44676babe1a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27d90d33b7f84b16b962b6bddba72047', 'request_id': 'req-5948d726-1568-4399-a8f8-44676babe1a5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:12:59.385678 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5948d726-1568-4399-a8f8-44676babe1a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27d90d33b7f84b16b962b6bddba72047', 'request_id': 'req-5948d726-1568-4399-a8f8-44676babe1a5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:12:59.392122 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5948d726-1568-4399-a8f8-44676babe1a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27d90d33b7f84b16b962b6bddba72047', 'request_id': 'req-5948d726-1568-4399-a8f8-44676babe1a5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:12:59.405478 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5948d726-1568-4399-a8f8-44676babe1a5 admin admin] The token cache is being invalidate because project 27d90d33b7f84b16b962b6bddba72047 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:12:59.405832 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5948d726-1568-4399-a8f8-44676babe1a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:12:59.406566 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5948d726-1568-4399-a8f8-44676babe1a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:12:59.408244 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1464/2917] 10.222.0.78 () {58 vars in 1232 bytes} [Wed May 25 15:12:59 2022] DELETE /identity/v3/projects/27d90d33b7f84b16b962b6bddba72047 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:13:08.999691 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:13:09.401983 dv-cinder-803823-9-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:15:01.484300 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af273b41-36e5-4c09-8eb2-625ea6c3c57d service neutron] Authenticating user token {{(pid=86692) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:15:01.485710 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-af273b41-36e5-4c09-8eb2-625ea6c3c57d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:15:01.555252 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af273b41-36e5-4c09-8eb2-625ea6c3c57d service neutron] Validating token access rules against request {{(pid=86692) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:15:01.615865 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54181ceb-cc93-403a-911e-5a8af6f52425 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63789c0a113447ff90933d447cf5bac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '908a13d2b49544fbaf760b07e213bb7d', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86692) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:15:01.616520 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54181ceb-cc93-403a-911e-5a8af6f52425 service placement] REQUEST_METHOD: `GET` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:15:01.616802 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54181ceb-cc93-403a-911e-5a8af6f52425 service placement] SCRIPT_NAME: `/identity` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:15:01.617075 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54181ceb-cc93-403a-911e-5a8af6f52425 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86692) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:15:01.618884 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.fernet_utils [None req-54181ceb-cc93-403a-911e-5a8af6f52425 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86692) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:15:01.630566 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54181ceb-cc93-403a-911e-5a8af6f52425 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:15:01.632473 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54181ceb-cc93-403a-911e-5a8af6f52425 service placement] RBAC: Authorization granted {{(pid=86692) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:15:01.642401 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: [pid: 86692|app: 0|req: 1454/2918] 10.222.0.78 () {60 vars in 1365 bytes} [Wed May 25 15:15:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:15:11.638090 dv-cinder-803823-9-iscsi devstack@keystone.service[86692]: DEBUG dbcounter [-] [86692] Writing DB stats keystone:SELECT=58 {{(pid=86692) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}